Google Coral M.2 on non-x86 GNU/Linux SBC

[DiTBho]

The Google coral M.2 Accelerator is a small ASIC designed by Google that accelerates Tensor Flow Lite models in a power efficient manner.

It's available in an M.2 module/E-key that includes two Edge TPU ML accelerators, each with their own PCIe Gen2 x1 interface.

- - -

anyone?
any experience?

- - -

tempted to buy one for my new MIPS/EL SBC 

[DiTBho]

Some Python examples, here (get started)

[DiTBho]

Required software components? here more information on building libraries, misc and kernel drivers, to run inference on the Edge TPU

Hardware, any constraints? Sure, being a strong interrupt driven co-processor, whichever SBC you choose, MSI-X is required by the Edge TPU Silicon Engine over PCIe, otherwise without it there's no way to get interrupts and therefore no way to know when operations have completed.

I see, 64 bit is also required as a hard constraint - won't this work on a MIPS32? Do you need MIPS64? 
(I think it has to do with large buffers allocated on the stack)

[SiliconWizard]

What do you want to do with it?

[DiTBho]

world domination .... something 

[SiliconWizard]

world domination .... something 

Good luck, you have a lot of competition out there already.

[DiTBho]

world domination .... something 

Good luck, you have a lot of competition out there already.

yeah, already too many people laugh, you have to exercise to laugh louder 

[DiTBho]

first crazy app: Imagine having your super secret router inside a full metal case in your camping tent, at a hacker camp meeting - do you really believe that your router disk is securely protected by an internal contact button as simple "tamper proof sensor?

Do you think it is better secured with a bicycle chain lock?
Do you feel it is better secured with 5 Kg of locks?

last time in the nearby tent they showed how to open all kinds of locks

TPUs rely on computer vision and visual deep learning at the edge, so your router can actually "see" who (a human being, but not you!!!) is looking around its metal case and run defensive algorithms(1) 


(1) send me a message on my Smartphone "Mayday Mayday, this is your router talking, stop hitting on the cos-player girl dressed as TombRider or Trinity or Captain Marvel (input from your taste profile) and get right back to the tent before the hackers crack my rootfs with tons of Trojan horses and all kind of root*hit !!!"



Planned for: next summer!

[SiliconWizard]

world domination .... something 

Good luck, you have a lot of competition out there already.

yeah, already too many people laugh, you have to exercise to laugh louder 

I would have posted a video about a real event rather than fictional to illustrate the point, but that probably wouldn't please moderation.
 

[SiliconWizard]

first crazy app: Imagine having your super secret router inside a full metal case in your camping tent, at a hacker camp meeting - do you really believe that your router disk is securely protected by an internal contact button as simple "tamper proof sensor?

Do you think it is better secured with a bicycle chain lock?
Do you feel it is better secured with 5 Kg of locks?

last time in the nearby tent they showed how to open all kinds of locks

TPUs rely on computer vision and visual deep learning at the edge, so your router can actually "see" who (a human being, but not you!!!) is looking around its metal case and run defensive algorithms(1) 


(1) send me a message on my Smartphone "Mayday Mayday, this is your router talking, stop hitting on the cos-player girl dressed as TombRider or Trinity or Captain Marvel (input from your taste profile) and get right back to the tent before the hackers crack my rootfs with tons of Trojan horses and all kind of root*hit !!!"



Planned for: next summer!

OK.

[DiTBho]

I would have posted a video about a real event

I don't have a video handy yet, but I'll probably record something.

Anyway, aside from the *playful aspect*, these things are made for catching bugs.

For example, two years ago, no one had noticed a terrifying bug on the PCI of a Linux/MIPS router until I had the sick idea of trying to get some weird PCI card to work.

[DiTBho]

MSI-X is required by the Edge TPU Silicon Engine over PCIe, otherwise without it there's no way to get interrupts and therefore no way to know when operations have completed.

While it's usually easy with x86-SBC, with MIPS and PowerPC SBC it's not easy to tell immediately if it's supported or not.

For MikroTik's hw things, Yesterday I opened a Ticket to speak directly with someone

[brucehoult]

Note the "not supported" and "doesn't work" are very different things.

Quite a few Arm and RISC-V SBCs have M.2 but it's usually M-key for SSDs. Some have an E-key for Wifi/BT, but the ones I've looked at only have 1 PCIe lane to them and the Coral needs 2.

[DiTBho]

Good point.
Which restricts the usable set.

[DiTBho]

Note the "not supported" and "doesn't work" are very different things.

With bug-fixing(2) I meant - doesn't work *even* if it's supposed(1) supported(3).

(1) because so declared by its documentation
(2) the most shocking discovery happened with the RSP, since the first bug was the documentation: it said nothing except "PCI 2.1 compliant", so I assumed "full PCI compliant" whereas it was rather "only partially implemented", and that's the annoying part with a lot of SBCs around and modern technology.
(3) with the latest router the documentation didn't say you have to "clear a bit in the controller" before issuing a PCI request, so the SBC was randomly pushing junk into memory with a probability of one in every million IOp  of disaster - > took weeks to figure out what was wrong

[DiTBho]

There are three version of Google Coral accelerators for SBC

• Mini PCIe Accelerator, datasheet
• M.2 Accelerator A+E key, datasheet
• M.2 Accelerator B+M key, datasheet

B+M is confusing, since

Key B = sATA
Key M = PCIe x2
Key B+M = sATA or PCIe

so why key B?!?

also, the datasheet says

Features
Google Edge TPU ML accelerator
...
PCIe Gen2 x1 interface

so, is it 1 lane?!? 

[DiTBho]

Key A = PCIe x2 and/or USB
Key E = PCIe x2 and/or USB
Key A + E = PCIe x2 and/or USB, but it depends on the internal wiring of the motherboard


=> more confusing => to be avoided if it's possible!

[MrMobodies]

https://coral.ai/static/files/Coral-M2-datasheet.pdf

This on-device ML processing reduces latency, increases data privacy, and removes the need for a constant
internet connection 

Excellent.
Looks to me like a nice little toy to play with.

[DiTBho]

so, after a lot of personal research, most non-x86 SBCs don't have MSI support over PCIe

it's really frustrating, datasheets don't even mention any detail about the PCIe.

[SiliconWizard]

Can it implement GPT-4 though?

[nctnico]

The Google coral M.2 Accelerator is a small ASIC designed by Google that accelerates Tensor Flow Lite models in a power efficient manner.

It's available in an M.2 module/E-key that includes two Edge TPU ML accelerators, each with their own PCIe Gen2 x1 interface.
anyone?
any experience?

This works just fine on Arm64 platforms.

[DiTBho]

This works just fine on Arm64 platforms.

64bit arm 

which SBC? with m.2? with miniPCIe?

[ejeffrey]

so, after a lot of personal research, most non-x86 SBCs don't have MSI support over PCIe

Really? How does that even work?

[DiTBho]

Really? How does that even work?

works but without MSI support.

[ejeffrey]

Really? How does that even work?

works but without MSI support.

My point was: MSI(-X) is the only way to do interrupts on PCIe.  There are PCI interrupt emulation messages, but it's all MSI based.  So either that means it has no interrupts at all, or it means something different than what I understand.  It's hard for me to see how a PCIe bus that doesn't support interrupts either meaningfully implements PCIe or supports much of anything.