Multiplexing I2S with SPI

[paulca]

Consider I have a set of I2S sources.  Call it too many for a single MCU or reasonable DSP.  Could be 6.  Eventually they are to be mixed, processed and then sent to one or more outputs.

A single MCU will not have the available number of I2S peripherals to use a single I2S per source/output.  So some form of hierarchy or multiplexing will be required.

I can see two options.

1.
Investigate and resolve this with multiplexed I2S using custom frames and the LRClock as channel strobe.  If STM32 I2S supports this mode without SAI.

An I2S 'frame' might consist of CH1L | CH1R | CH2L | CH2R.....

In theory this should work up to the bitrate of the underlying SPI peripheral.

EDIT:  The issue is that the streams are generated from multiple MCU front ends.  Multiplexing would only be able to combine sources for the same MCU.  It's still a "hierarchy" options which might be enough.

2.
Just offload the I2S data buffers as bytes on a SPI bus.  Basically TDM it.  As this approach removes the need for LRClock and the master clock (for now), it "seems" simpler.  Effectively I'm bringing maybe 15 or 16 data lines down to 2 high speed ones basically. I can use the DMA Transmit+Receive duplexity to receive a "packet" from the slave, while sending the last processed packet to the output bus.  Giving me a tri-buffer IN, PROCESSING, OUT and a 2-3ms latency.

I ran an experiment and between an F411 with an I2S source (48K@32bit) and an H743, it took 230us to send a one millisecond audio 'packet' 192 32bit words.  I have to test 16bit, but still, it's not like I am getting 20:1, it looks like the bus will be a lot more crowded than that.  If I use 16bit I2S I 'might' get 8 on there, 6 should be ok.   The limitation is the 50Mbit/s SPI rate on the F411.  I could push that to maybe 80Mbit using a F407 but they are harder to find these days.

Buffer alignment for mixing them together shouldn't present too much of an issue if it's done right.

Pseudo-code

Code: [Select]

Handle EXT Interrupt:
  if isOurCSLine AND ourCSLineIsLow:
    for each channel:
      if channel isTxready:
        SPI_DMA_Transmit channelBuffer
      else
        SPI_DMA_Transmit zeros


Code: [Select]

  wait until some magic timer we haven't worked out yet tells us it's the start of frame/window
  rotate buffers
  Lower Slave 1 CS line.
  SPI Receive N channels of data OR Timeout in N clock ticks
  If Timeout Alert and redirect to zeros
  Raise Slave 1 CS Line

  Lower Slave 2 CS line.
  SPI Receive N channels of data OR Timeout in N clock ticks
  If Timeout Alert and redirect to zeros
  Raise Slave 2 CS Line 

...

  Mix Channels to Output

I'm still thinking out loud.

Maybe there is a better way.

[SiliconWizard]

Not sure I completely understand what you are doing.
If you have full control over your audio data sources (to the point of being able to use "custom frames"), why use I2S then?

[alexanderbrevig]

Did you consider an FPGA?

[paulca]

Not sure I completely understand what you are doing.
If you have full control over your audio data sources (to the point of being able to use "custom frames"), why use I2S then?

I have control over what the front ends do with the I2S stream from a bit of hardware; USB interface, SPDIF bridge, DAC, ADC.  ie.  I can resample, repackage etc.  One USB interface sends it's 16bits on a 32bit frame the other on a 16bit frame.  The front end undoes that for example to standardise the internal.  It allows me to decouple from what a random bit of hardware gives me.  The idea was to create a modular architecture where front ends and outputs could just be added to the SPI bus either side of the main processing MCU.

I looked into SAI and it supports multi-slot I2S.  Although the H743 might have enough direct hardware for all of my sources and outputs while using SAI.  It has 4 dual block SAIs, each capable of 2 I2S streams.  Assuming I can get them to work, that's 8 and after I tested a a CubeMX setup with all 8 configured the 3xI2S are still available.  If I can get it to mix those into 2 output streams and run a 5 band EQ on both of them, in one MCU that might save a lot of hassle.... or make the hassle on that one MCU a lot worse.

I figure I'll test it, see what I can get the single H7 to do.

To be honest the MVP is for 2 USB + 2 Line out.  If I get there it will be useful to me.  BT in and out and Optical in and out can be added later.  Maybe instead of aiming for the hills in the distance I should see what I learn doing the MVP.

[tszaboo]

What's wrong with digital (or analog) multiplexers for this? 74HCT4053 or something like that?
They might pop when the switching is happening, sure, maybe if you do a break before make, or mute the DAC while switching its avoided.

[paulca]

What's wrong with digital (or analog) multiplexers for this? 74HCT4053 or something like that?
They might pop when the switching is happening, sure, maybe if you do a break before make, or mute the DAC while switching its avoided.

Sorry.  I should have explained, but didn't want to drone on.

The sources and outputs are simultaneous, not selected or routed.  The sources are not (necessarily) music or media, but rather "interface" and "notification".  Specifically I have several desktop computers, each of which has audio output.  I also have audio from my phone, from projects, etc.

I also have wireless audio, phone, headsets, headphones.

For outputs I currently have a set of desktop speakers, a custom wired headphone amp, several wireless headphones.

The problem I am trying to address is enabling the ability to have some or most of those audio sources active and the ability to "send" them to any or all outputs.  Without requiring any one of them to be on, present or active for it to work.

For example, I'm using the little low power desktop at the moment, because I'm "in work" on the work laptop, I'm not going be needing the 110W power hungry gaming PC to be on.  So I have to, each morning, unplug the audio from the main PC and plug it into the desktop and accept all the crap noise that introduces.  WHen I finish work and want to game I have to switch back.  Similarly the work laptop has audio output which would be "nice to have" always routed for notifications, even if I am listening to extremely loud music from the other PC.  I'd still like that "Ding Ding", when the boss phones me on teams.

If I ran the requirements through a methodology like DSDM/Agile and Moscow I "could" accept initial versions require some switching and are therefore not N->N simultaneous.  I could simplify the design a lot by focusing on specific use cases for example.  I mean, it is actually extremely unlikely that all inputs will be active and assigned to all outputs.  Extremely unlikely.  However, I prefer to let designs form themselves, so I start with very loose but ambitious requirement.  This is what spawned the idea of normalizing and multilpexing all the inputs onto 2 internal buses.  Using one or two beefy ARM micros to do the EQ /processing on those and again output them on one output bus.  The outputs then complete the sweety wrapper pattern by feeding off one bus.

Inputs are gained and assigned (by the user) to Bus A or Bus B.  Then get processed by Processor A or B and output on output bus A or B respectively.  Each output then can select Bus A, Bus B or MUTE... and final gain.

That one line sums up the design.  Throw a bunch of cheap MCUs at it, each having a relatively simple task, spread out the load, "bus up" for speed.  Adding a switching layer such that, say, only 1 of each pair of I2S inputs on a front end is active at a time and I just use a micro + multiplexer or line switcher circuits as you suggest to digitally multiplex them based on user input of which is active.  It would lower the number of streams having to be shipped to the processing bus.  Lowering the number of I2S streams to marshal... removing the need for SPI.  Would lower the BOM too a I could use a single F401 or even an F103 to manage the line switching for all 6 or 8 inputs.

To be honest, one thing the F411 had going for it was it could provide a USB Audio interface itself without additional hardware.  However after much fighting and research I decided that STM32 USB libraries are ... well, life is just too short for that TBO.  Using a hardware USB->I2S will hopefully be a lot simplier, it seems that way so far.  So using F411s as front-ends lost that advantage ... could be on the cusp.

What I'm resisting is taking the easy turn off the road to my ultimate goal by accepting something that "will do" or "will do 90%" of what I wanted, but it was just easier.  I'm resisting it, because I know I'll just drop the project and the temporary permenant solution will remain.

It's bound to end up the case that a primary source like an optical spdif from the main PC is paired with an auxilary input like Line In... and I will want both on.  I can stagger and arrange pairs, but I expect it will actually increase the number of front end pairs to get N of type -> N.

One worry about trying to pack it all into one STM32H7 MCU is that the firmware problems involved in doing it all together in one place could get pretty messy and thus buggy.  Splitting it up into "componentised" microcontroller sub modules allows me to divide the problem into smaller chunks, even if it requires a bit more work and considerably increase over minimal BOM.

[paulca]

In the background I am looking into just the single H7 solution using I2S via the SAI peripherals.

At the time however I already have a breadboard with 2 I2S going through an STM32F411 onto SPI to an H7.  I've only tested so far as recieving the timing a single 192 byte transfer.  The 270us threw me.

What threw even more though.  There is no HAL timeout value shorter than 1 millisecond (AND it's no accurate, it's +/- 1ms).  So the instant a slave fails to responds for whatever reason it crashes the whole bus.  I'm sure that is fixable, but it require overriding a few HAL functions.

I mean I have "safeties" on both ends.  If the slave has nothing to send, it will send 0s.  This is meant to make the bus multiplexing synchronous and fixed bandwidth to simplify timing.  It still leaves the risk of a stalled, crashed or otherwise not available slave to stall the whole bus out.

I even at one point got really ambitious and even got as far as trying to code a "mutal consent bus mastering" solution where slaves where optional, if a slave had something to send, it would "grab" the CS line and hold it down after the master tugged it.  So if an endpoint was inactive the master would tug the CS line for 8 SPI clocks and release it, the inactive slave would not hold it down, so the bus master would proceed to the next slave.  It turns out that kind of bidirectional signalling at those kind of bitrates, using bi-directional GPIO comms is... well, lets say it's not the easy solution.  As soon as you start having to add "grace periods" for timing to allow slaves to respond and ... your 1 milisecond bus budget is busted.

Why 1ms?  Why not.  I'll take anything under 10ms with a laugh and anything under 20 or even 30 could be tolerated.  But nothing has, yet, told me I can't use 1ms buffers.  SPI timing, if I persist in TDM bus'ing with SPI, would get a lot easier with a, say, 8ms buffer.  I could afford to have a 1ms timeout and still recover.