AI(machine vision/obj detect): Hailo-AI by MS

[DiTBho]

I don't know if it is already available for purchase, my colleague received some hailo.ai samples.

A couple of Mini M2 modules, which somehow sound like an alternative to Coral-AI 

Intrigued, I searched for documentation(1) and found some already published examples here (rpi5).

In practice, talking about the machine vision class, it's an accelerator for real time recognition of complex objects such as people, cars, trucks, etc

(1) I am not planning to use it on a rpi, but rather on a MIPS(2) based router.
Anyway, found documentation here (ai-kit) for rpi5.

(2) only MIPS32r2/le, maybe also LoongArch, but no chance to make it running on MIPS4, MIPS5, MIPS5++, Loongson

[DiTBho]

Anyone ? 

 

[SiliconWizard]

Yes as you hinted, there's already a "hat" for the RPi. https://github.com/hailo-ai/hailo-rpi5-examples

You probably realize that you're the only one trying to port brand-new tech to MIPS platforms.

What do you want to do with AI on a router?

[DiTBho]

a "hat" for the RPi

it's basically a M.2 PCIe expansion.
If a SBC already has it, you are ready.

[DiTBho]

Anyone ? 

Regardless of the ("nonsensical", LOL) things I am doing: is *anyone planning to use* this solution for projects?
Any type of hw is allowed, even an RPI5! 

[DiTBho]

none 

[Bud]

Solution looking for a problem ?

[DiTBho]

Solution looking for a problem ?

first you have to make the module work, easy on RPIv5, not easy on other platforms, and before that, meanwhile you have to study how the code works, and it wouldn't even be bad to rewrite the support to take advantage of more NPU modules.

[Rijad12]

We can buy Hailo-8™ M.2 2280 module at eShop of AAEON Technology Inc.
However, Hailo-8™ M.2 2280 module has possible 26 TOPS performance and is hence more powerful and more expensive than Hailo-8L which is sold as part of Raspberry Pi hat.
You can see both here: https://hailo.ai/products/ai-accelerators/

[DiTBho]

Hailo-8™ M.2 2280 module has possible 26 TOPS performance

I'm working on integrating it into a MIPS-based project.
I don't know how many people buy it, and if it's mostly sold for RPI-only, but I'd like to get both.
At worst I use RPI to "copy" ideas and working-code, then replicate it on other platforms.

[Rijad12]

It is not restricted to Raspberry Pi.
I put that Hailo-8 M.2 Module to M.2 slot instead of my M.2 SSD, and it recognized Hailo-8 on PCIe interconnect (I was booting Linux from USB stick instead of SSD).
Hailo-8 has Model Zoo (look at their Github account) with some examples.
Maybe it would be resctrictive that for training of neural networks you need some kind of Debian-based Linux running on Host-Computer, with either x86 or ARM based platform. Other platforms like MIPS or RISC-V are for training not yet supported. I do not know for inference yet, if there are less restrictions.

[DiTBho]

It is not restricted to Raspberry Pi.

I didn't say it. I said it needs support. x86 and RPI already has it "kernel-in-tree".
On MIPS I have to fix stuff from the ePCI up.

Debian-based Linux running on Host-Computer

Python needs some extensions on Gentoo/MIPS and HPPA. This is the second type of thing I have to "copy" from Debian, Raspbian etc.

-

What is you final application?
How much did you pay for your module?

[Rijad12]

I was thinking if it could be used in some industrial camera, but I am only experimenting with it so far.
Hailo-8™ M.2 2280 costs 220 USD at AAEON Shop at the moment.

[quince]

Hailo-8 is $200 for 26 TOPS at UP Shop: https://up-shop.org/hailo-m2-key.html

Does anyone care to share the memory capacity of this chip? Pretty important figure. I can't find it anywhere: https://www.google.com/search?q=hailo+8+sram

The Coral TPU only had ~8MB of SRAM for caching weights, so it's pretty hopeless to run inference with models bigger than 4-6M weights. https://www.google.com/search?q=coral+tpu+sram

If they're leaning hard on the PCIe3x2 / x4 interface for streaming weights, I don't see how they ever hit 26 TOPS in practice.

A useful metric would be, how many weights is the largest vision model they can run on there, and how many FPS does it hit.

[SiliconWizard]

To paraphrase a famous singer:

Baby, I can feel your Hailo
Pray it won't fade away