USB- Control Signal and Isochronous video stream

[sadasivamASK]

Hi,

We are using iMX8MINI EVAL board for streaming video to Android mobile via USB-OTG cable.

From my understanding, "Isochronous" transfer type can be used for Video Streaming. And "Control Signal" Transfer type can be used for sending control signals.

My doubt is like,
1. During video streaming, can we able to send any control signals via USB-OTG?
2. If we can, does any delay or interruption occur during video streaming?

Thanks,

[moffy]

Enumeration and control signals are established via USB through endpoint 0. This is where you configure your Isonchronous Video endpoint which will be established as some other endpoint number. I am not sure, but if the Video needs its own control signals, which would be necessary for play/pause etc. commands, you would need to configure a Control Endpoint for the Video, apart from the actual Video streaming endpoint. They are separate packets going to separate endpoints so they don't interfere, like TCP/IP packets sent to different ports on the same machine.
My understanding of USB OTG is that it can act as a HOST or DEVICE but once configured it's just USB.

[Sauli]

The video class device specification defines all interfaces/endpoints you need to control a video class device. It is a complicated specification, at least for me too complicated to fully understand.

If you want to send/receive control signals unrelated to the video, you can always use a vendor specific control transfer to endpoint 0, targeted for the device. As long as there is USB bandwidth available these transfers do not disturb the video.

[sadasivamASK]

In USB-Spec,

The specification places limits on the bus, allowing no more than 90% of any frame to be allocated for periodic transfers (Interrupt and Isochronous) on a full speed bus. On high speed buses this limitation gets reduced to no more than 80% of a microframe can be allocated for periodic transfers. So, from this point, Can we say like, the Bus is having enough bandwidth for sending control signals.

[sadasivamASK]

So, this is possible that we can go with USB protocol (considering the bus limits- 10% will be available) and should not go with any customized USB protocol.

[moffy]

Can't imagine you getting video down a full speed bus, the high speed bus with the 20% allocation or (96Mbits/s) for control will be heaps. The interrupt rate or rate at which packets are sent is 1kHz for full speed and 8kHz for high speed, which gives a 125us latency for the high speed bus.

[sadasivamASK]

Hi,
Then how they telling 10% for control signals which wont affect the video stream

[moffy]

I have no idea. I suppose you could get a highly compressed video stream down, but I wouldn't like to try it. The 12Mbits/s full speed was meant for peripherals not video. Audio is possible too but that is limited to 96ks/s stereo on full speed you need to go to high speed for the higher rates Audio 1.0 vs Audio 2.0 standard.

[Sauli]

Isochronous and interrupt transfers combined are allowed 90/80% of bus bandwidth. Bulk transfers are not guaranteed any bandwidth. That leaves 10/20% for control transfers. Control transfers are not meant for transferring any significant amount of data, so probably the there always is free bandwidth within that 10/20%. Even if there wasn't, it is the control transfer which is delayed, not the isochronous video stream.

We are using iMX8MINI EVAL board for streaming video to Android mobile via USB-OTG cable.

You are aready streaming the video? In this case adding some control transfers will give you no problems.

You can transfer low resolution raw video through USB 2.0, and better quality compressed video too. I am no expert on this, just looked up the subject on the internet.   

[sadasivamASK]

Is it possible, can we send custom command from iMX Device  to Android Mobile

Since, from Setup Packet of Setup Transaction, Std./Vendor/Class specific- Requests are available to give control commands.

For Custom commands like - Video Pause, Video Record how can we implement?

[moffy]

Which device is going to be the Host and which the Device? Normally you configure the iMX device as one or the other, there should be plenty of examples in the SDK, and the Android device will configure itself as the other once plugged into the iMX, assuming it has the appropriate drivers and recognises the iMX, that is often the catch. This quite possibly isn't the best forum to ask these questions, you might be better with an Android developers forum.

[sadasivamASK]

Here, iMX will be the host which initiates the video stream to Android(which is acting as Device). During video stream, custom control commands will be sent to control video stream. How can we implement custom command for controlling video.

If we go with Video Class Definition, there is no control for recording or image capturing and all... So, we planned to customize the USB by giving cmnds and modify in Android too to meet.

[sadasivamASK]

Either, if we go with Audio/ Video Class definition or Vendor specific transfer type, anyhow control byte will be given in Data Transaction of Control Transfer type which can be captured in Android(Android will do the response accordingly). So, it is possible to send custom commands during streaming right?(using Audio/ Video Class (or) Vendor Specific type)

[moffy]

You really need Android developers to answer your questions.

[sadasivamASK]

If we configure iMX(Host) as Audio/ Video USB Class, Can we give user-control command(like E0-video capture, E1-Image capture)
in "Data Transaction" stage of Control Transfer type.

My understanding in Control Transfer type for user commands:

1. Setup transfer
2. Data transfer                      <<<---User-control command
3. Status transfer

Is it possible, the User-control command send in Data transfer phase?

[Sauli]

Here, iMX will be the host which initiates the video stream to Android(which is acting as Device).

Then, the iMX is not a Video Class Device, and you should not even try to implement that class on the iMX. I don't even know what device class the Android assumes in this case - a vendor specific class perhaps?

Is it possible, can we send custom command from iMX Device  to Android Mobile

If the iMX is the host, then it sould be easy. Just make a vendor specific control transfer with any data you wish to pass. But the Android end must have code to process that transfer.

If things were the other way round (Android host and iMX device), then the iMX could be a Video Class Device. I have no idea if an Android phone as host supports this class. Why should it - the phone has a very good camera of its own. Sorry, I assumed you had a camera connected to iMX, but you never said so!

In general, if you want to send custom control commands from host to device, the device must have code to process them. If you want to send control commands from device to host, the host must poll (send IN requests to) the endpoint used for those commands. The endpoint used could be the control endpoint 0 or a dedicated interrupt endpoint. Anyway, a custom command requires dedicated code on both host and device.

Finally, I cannot give any advice about Android. I do not have the skills to write the simplest 'Hello World' program for it.

[sadasivamASK]

Thanks Moffy and Sauli for the efforts and inputs given at this crucial time. Thanks for the support....

Camera Module is attaching with iMX...

Also clarification needed on below query:

In Host End, how can we give user-control command(like E0-video capture, E1-Image capture).

My understanding in Control Transfer type for user commands:

1. Setup transfer
2. Data transfer                      <<<---User-control command(E0/E1)
3. Status transfer

Is it possible, the User-control command send in Data transfer phase?

[Sauli]

Is it possible, the User-control command send in Data transfer phase?

The simple answer is yes.

In the Setup packet set request type in bmRequestType to 2 (Vendor) and wLength to the length of data you are going to send (no more than maximum packet size).

For example in Windows you can do this with the WinUsb_ControlTransfer() API function. I guess Android has something similar. The iMX may have USB host examples which no doubt do control transfers.

[sadasivamASK]

Yes, Thanks Sauli...

[sadasivamASK]

By using the same approach can we able to control host by sending control commands.

My understanding is like when USB OTG is connected, Android mobile will acting as a host and iMX will acting as a device.

I provided the info like imx(host) and Android(device) considering data transfer trigger. And, moreover we provide supply to iMX from Android mobile. So, is it possible can we control host using USB Device via control commands.

[Sauli]

So, is it possible can we control host using USB Device via control commands.

Yes, but since the device cannot initiate a data transfer the host must be constantly polling the device. For hardware button press events the UVC class specification has the interrupt endpoint. Compared to using a vendor specific control transfer through endpoint 0 this has the advantage that the device does not constantly have to process SETUP packets. Only when there is a button press to report, you prepare new data to be sent to the host on the next interrupt IN transfer from the host.

If you insist on using control transfers for getting device status, I suggest you set up a timer (e.g. 10ms interval) to send the control transfer requests from the Android application.

[sadasivamASK]

Yes. Thanks for the support. Will work on it and let you know.