Linux Dependency Black Hole

[ataradov]

MCUXpresso officially supports only Ubuntu 20.04.2 LTS / 22.04 LTS. They can't guarantee operation on Debian or any other OSes. It does not mean it will not work on other variants, but if you try, you are expected to figure it out on your own. There is no reason support should waste time trying to figure it out.

[shapirus]

I also think things are getting worse. The reason are all those pesky new/better/wonderful ways to install software... that are full of shit. They don't want to learn how to create debian packages, or think they will make some money going that way, or fame/glory/whatever, so they have to invent the wheel again and again. But, while apt is almost the perfect wheel, those new wheels, well, not so much.

Apt is almost perfect, yes, but only as long as you install the software available in the distribution's repositories and, ideally, don't mix the distro releases (that can be done too, but requires experience and understanding).

The core issue with, almost universally, linux package management, is dynamic linking. Once you want to install something that was linked with a library that is not (or no longer is) a part of your distribution, you're screwed. That's where hell begins. It's good if there's a source .deb package: you can usually build a binary package from it using dpkg-buildpackage, and it will be linked against your available libs, and will install just fine. It's worse when you have plain sources. It's terrible when you have just binaries linked against libs in a 5 year old distro.

This is where packaging systems like AppImage can be helpful: they contain statically built binaries with no external dependencies. They will still work in 20 years just like they do today (provided it's the same CPU architecture), unless they depend on some specific kernel calls, which is not typical for general userspace apps. Good luck trying to run anything built for any Linux distro in 2005 today, unless it's a static binary.

I can see why software developers want to bundle their binaries as AppImage: spending time to learn how to build a gazillion different packages for a gazillion of different distros, not to mention another gazillion of their releases, would be insane. Why do that if you can build a universal package (actually several: by the number of target CPU architectures) that will be guaranteed to work on any linux distro?

Linux packaging is perfect as long as you stay strictly within a given distribution's ecosystem. Once you try to take a step outside (and you will, unless you have a purely academical interest in it), it becomes a nightmare.

[tatel]

Apt is almost perfect, yes, but only as long as you install the software available in the distribution's repositories and, ideally, don't mix the distro releases (that can be done too, but requires experience and understanding).

(...)

Linux packaging is perfect as long as you stay strictly within a given distribution's ecosystem.

I agree. However, I have to say, once a package for Debian is available, one can expect it to be incorporated by the gazillion distros that follow Debian. I wouldn't like to have all my applications compiled statically, just because wasted space. But, yeah, good for just a couple of things.

Once you try to take a step outside (and you will, unless you have a purely academical interest in it), it becomes a nightmare.

I understand what you say, but can't really agree here. I'm used to build what is not in the repositories. I find there's a couple of advantages coming from that, like optimizing for you processor and maintaining control over your system. But, as said, I get anything back to Debian's ecosystem ASAP. Not worth the hassle, usually. Can't really understand what you mean about "purely academical interest". I did my first Linux install on a brand new PowerBook G3 in the past century. Since then, I know quite a few people that doesn't need anything outside Debian's ecosystem to do his daily work.

Now, to remain on point with OP, I see that pyenv is a tool to manage different python versions. MCUXpresso is a software developer suite from a  vendor. None of these are SDR software. I was under the impression that's was the target, to play with SDR software. SDR's not my thing, so I don't know. But I think there are quite a few official Debian SDR packages that could perhaps be useful?

https://blends.debian.org/hamradio/tasks/sdr

[mag_therm]

Hi tatel,
The install of MCUXpresso was failing on "can't find Python3.8"  So one of my questions was "Why does it specifically need 3.8, when higher versions of python are already installed in their correct paths."
So I had to install pyenv which allows download of any python3 version.

What happened today, there is a new version of MCUX .deb.bin in NXP along with updated manuals ( I had downloaded the old version about 3 days ago )  I  have just downloaded new version now.

[tatel]

Well, I would say, if python >3.8 could be used, then the "vendor" is to blame for bad packaging.

I guess you are looking to use some specific software for your own reasons.

If that's not the case, then I would look at the hamradio task. You can click the names on the previously given link to see a description. Perhaps there's enough stuff there to make you jump over a couple steps. E.g, I see there's a fork, jtdx, with a high enough version number, of that WSJT-X thing that is driven metrologist nuts on that other thread. I think the stuff in that link it's worth a look. It could all be installed by just ticking the "hamradio" task during the install.

In my experience, it's better to stick with pure FOSS than to try and use some software coming from a for-profit source. Many times, it's not as easy and doesn't have all the features of privative software. FreeCAD comes to mind almost immediately. But in the long run I think that's often worth the hassle; it will not vanish when the vendor gets bought or goes out of business. Particularly if you aren't going to use it professionally. OTOH, GRASS GIS was the thing that got me into Linux: it was able to do much more than any privative GIS software we could afford then. YMMV.

[JohanH]

The install of MCUXpresso was failing on "can't find Python3.8"  So one of my questions was "Why does it specifically need 3.8, when higher versions of

I've had success with some proprietary software by creating a symlink to the newer installed version in the place where the software is looking for the version that it supports. Your mileage may vary...

[tatel]

The install of MCUXpresso was failing on "can't find Python3.8"  So one of my questions was "Why does it specifically need 3.8, when higher versions of

I've had success with some proprietary software by creating a symlink to the newer installed version in the place where the software is looking for the version that it supports. Your mileage may vary...

Yep.

But, if the vendor isn't able to do that... well, that speaks volumes

[xvr]

> What happened today, there is a new version of MCUX .deb.bin in NXP along with updated manuals

Do you known exact distributive and version of Linux on which MCUX could be run?
If yes:
1. Install Docker on your machine
2. Run known-to-work Linux version inside it
3. Install  MCUX .deb.bin inside it
Profit ...

[ataradov]

"Why does it specifically need 3.8, when higher versions of python are already installed in their correct paths."

Because their QA team tested it with this version of Python. And their support teams are ready to assist with this version of Python. They are not focusing on broad OS support, they are focusing on minimizing support tickets.

This is the version packaged with the specified supported OSes. This is how commercial software with official support works. Open software can afford to make some assumptions about compatibility, since there is no explicit expectation of support.

[mag_therm]

 OK.... On Fedora 37, the new version of  MCUX installer is now running though to completion (except it erases the bin after it finishes, ) leaving the problems
There are a few dependencies not found.
I have been able to clear a few, and there are about  8 or so still not found.
There are two reasons I think
   Fedora name suffix does not match
  Fedora path does not match
libncurses5, libncursesw5, dfu-util, libusb-1.0-0-dev

The problem with python3.8 was not that the libfile was" not found," it wants python3.8 to be active.
After I realized that, I was able to use pyenv to set 3.8 as system active

It is not worthwhile continuing with the dependency problems.

I am expecting the Debian installer to arrive.
If Debian does not work I will revert to Ubuntu

[mag_therm]

Hi tatel, thanks for the debian link.
I have installed some of those from Fedora32 repo, and some others from git tars. I recall there were  dependency issues on the tars, but eventually solved one by one.

the reason I need NXP MCUX IDE is for my new HackRF1 ( FOSH/FOSS)  transceiver.it has a NXP LPC series MCU.
I discovered that while the receiver is very stable, the frequency setting is an approximation. Probably due the the developers not being hams, the settings are neither monotonic nor accurate
I want to try to improve the math approximations  in the firmware for my own use, hence the NXP IDE
Regards
 

[tatel]

I'm not a ham, either.

It seems there's at least one hackRf related package official in debian, with command-line tools. I'm sure you could do your developments using pure FOSS tools, but you probably are used to some different thing? That would be understandable.

It there could also be some tool to convert packages from one distro to another, but IIRC, "alien" works the other way: RedHat->Debian

Ataradov said the software you are looking to use is for Ubuntu 20-22, and out of that, you are on your own. I think he's right. So, having Ubuntu installed (brrllrrll) would be the easier way. Short of that, it will probablybe  easier get it working under Debian that under Fedora.

Good luck

[mag_therm]

Hi tatel,

I wanted to get MCUX on Fedora because I thought (perhaps in ignorance, and should have asked)
that it would be the proper and maybe only way to access and update the firmware on the LPC4320 MCU in the HackRF1.
The firmware is updated like this:
 hackrf_spiflash -w hackrf_one_usb.bin

MCUXpresso  has been a nuisance with dependencies, and it is going to be another nuisance working outside of Fedora,
as everything here including all computers and the ham radio station runs on Fedora.

I would like to ask " How would a pro developer do this, do they just accept setting up a dedicated o/s to work on a particular MCU? "

I have started to examine the HackRF firmware math in C that sets up the frequencies for the Si5351 clock.
Not sure, but I can see two likely areas where approximations are made. one is integer divisions by prime numbers, and one is a frequency homing function that iterates until a result is in a tolerance.
 It is likely that approximations were made because the HackRF1 covers a very wide frequency range of 1 MHz  ~ 6000MHz.

So I have intention of building a firmware version for myself with a limited frequency range ,  say  1 ~ 200 MHz and better accuracy to 1 Hz if possible.
The reason for accuracy for hams on HF: I have been collecting data for ionospheric research ( HamSci) requiring ~ 10 milliHertz resolution and also the digital modes are being used with frequency lock to GPSDO , ( and new mode FSTW4)

I have corresponded with github GreatScott, with confirmation of the frequency selection accuracy limits and that there is no feedback of the actual frequency calculated..

[macboy]

I am not a Linux dependency expert, but I have learned a few things over the years. First: use a good distro. The various libraries included in a distro release are tested to work together. So if something needs XYZlib, then when you install XYZlib using the distro's tools (apt, etc.), it will not break the system. Unfortunately, this version of XYZlib is probably more than a year old - maybe a few years - and might not satisfy the requirements of the tool you are trying to install or build.

You can try a chroot or one of the many variations of this. This allows you to install an complete alternate root filesystem, which still runs on the present kernel, but is isolated from the system's main root filesystem. You could install the same system (but different packages/libraries), or a different release of the O/S (Debian 10, on a system running Debian 12), or a different distro (Arch Linux on Debian system), or even a different architecture (Debian for armhf on a Debian x64 system), in combination with qemu. The point of doing this, is that you can install a minimal O/S, then install only the dependencies for the application in question, then try to resolve the dependency issues. Since the number of installed packages is much smaller, the dependencies to resolve will be fewer, hopefully. This also avoids breaking the main system by installing some different version of something outside of the distro package system.

Alternately, you can try building the application with static linkage, so that it does not depend on installed shared libraries at all. Instead, all of the shared library code will be built into the executables. The dependencies are installed only for compilation (e.g. XYZlib-dev). You may/will choose to do this inside a chroot as well, to isolate the installation of the many build tools, dev libraries, etc. from the main Linux system. After the app is built, installed, and tested, you could delete (or archive) the chroot to save space.  The static-linked binaries will be bigger, sometimes much bigger, but as a bonus they usually run faster as well.

[tatel]

Hi.

Please don't take too seriously what i'm going to say, because I don't have any hamradio/MCUexpresso experience. To me expresso is a good way to make coffee.

You have hackrf_spiflash and other related command line tools on package "hackrf" officially supported on Debian, and I would bet my last euro they don't use that expresso thing to build that tools. Don't know if there is some firmware available. You'd have to look for it, or look at how firmware is loaded and to write your own firmware accordingly.

Most FOSS software is built with GCC and related tools with the usual letany "configure-make-(become root)-make install". That means there will already be Makefiles to build the hackrf utilities, the firmware loader, etc. You probably could modify and use these Makefiles. That way you can have a good peep at how things are done in the FOSS world when for-profit companies are not involved.

I would look at source packages for hackrf and anything you could be interested in. Probably you'll find quite a good base to start from. If you are not used to GCC way of things, perhaps you'll find easier to use another thing and avoid learning that FOSS way to build.

I can say that having knowledge about the GCC way could very well be worth the hassle. I would say most of linux software is built that way, and you could always get from the repositories a GCC ecosystem to be installed in your system without any dependencies problems whatsoever. Look for the Debian guides; if you start from debian source packages, you could easily even find that most of the work to produce debian packages is already done. Not that you need to create any debian packages to use the binaries you just built with GCC.

Best wishes

[tatel]

Another thing, developing tools, libraries, etc, take a big chunk of hard disk space, so you could find your Pi doesn't have that much space available. You could perhaps add some hardware, or do the development on a regular PC. In that case you would be creating binaries for an architecture in a machine with another, different architecture. To do that, you should look at "cross-compiling"

It basically means you first built a toolchain able to do what you want. That's usually how OpenWRT is built, and I'm pretty sure it will be the same for the Pi.

[mag_therm]

Hi tatel, That is interesting.
Yes I have downloaded the firmware  C files from git  and have been examining them. But I dont have C IDE or debug, (And not used C for 36 years, so a lot to learn)
So thought MCUX and a devel board  would give me everything including debug.

Also I have been thinking today, with the ease of uploading the firmware, and as I only need < 10 frequencies I could just build 10 firmwares each with a fixed dataset for the Si5351.
A little UI and bash to replace firmware on frequency change , and with the latest Gnuradio ( python based)  which allows users to build their own DSP,     could be an easy solution at least initially

[mag_therm]

Hi tatel, I don't yet have an R Pi here, I use industrial computers running on 13.8V DC , the main one is Aaeon 6651 https://www.aaeon.ai/en/product/detail/boxer-6641
No problem with capacity.

[tatel]

Hi tatel, That is interesting.
Yes I have downloaded the firmware  C files from git  and have been examining them. But I dont have C IDE or debug, (And not used C for 36 years, so a lot to learn)
So thought MCUX and a devel board  would give me everything including debug.

Also I have been thinking today, with the ease of uploading the firmware, and as I only need < 10 frequencies I could just build 10 firmwares each with a fixed dataset for the Si5351.
A little UI and bash to replace firmware on frequency change , and with the latest Gnuradio ( python based)  which allows users to build their own DSP,     could be an easy solution at least initially

Yeah, even not being a ham, that looks good. Using GNU-whatever looks right to any linux talib 

Using an IDE to build just a firmware binary file looks like using a battleship to go after a trawler to me. YMMV. You could have a look at this:
https://opensource.com/article/19/7/introduction-gnu-autotools.

Probably even a one-liner calling gcc would be enough for that.

Debugger: https://sourceware.org/gdb/

If you are so inclined and absolutely need to have an IDE, you probably could use Eclipse anyway, but then there you go again installing something not in the Debian repositories?
https://wiki.debian.org/Eclipse
https://www.eclipse.org/downloads/packages/release/helios/m7/eclipse-ide-linux-developers

Hi tatel, I don't yet have an R Pi here, I use industrial computers running on 13.8V DC , the main one is Aaeon 6651 https://www.aaeon.ai/en/product/detail/boxer-6641
No problem with capacity.

Oops, got that wire from metrologist, I'm afraid.

[mag_therm]

battleship to go after a trawler

It is 28 of .c and .h files. I wanted to put breakpoints to trace through the math which is meandering  over at least 3 of the .c
And the headers are all mixed up hard for me to follow I think due to multiple versions over 10 years.
Can all that be done in eclipse and gdb?

And Thanks for your time on this.

[tatel]

28 files?  There goes my guess about compiling just one file. I should have thought twice about it.

At this point you are going further than I have ever been. I was used to, say, manually patch realtime extensions into a kernel version that wasn't the intended one (when realtime patches were still not merged into mainstream kernel). So automatically patching the source would fail, but one could find the right places to patch manually by just looking at the code. There was usually a whole bunch of files to patch, but it was quite an easy task that called for very little programming knowledge.

I'm into system administration, not programming. However, you are about to code/debug, etc, by yourself. You'll need to learn more about programming than I currently know. So take what I'm going to write with a grain of salt. Others in this forum will know about programming much better than me, perhaps they could help from this point forward.

I guess it could be done in eclipse and gdb, yes. But to learn about how the builds work, I would:a) learn about autotools, then b) look at the source code.

I think that would be the fastest way to learn. I have never, ever, used an IDE for any build. Software was built that way, much before anyone could heard anything about Eclipse. And Eclipse (I guess) will use the autotools infrastructure anyway.

Usually just getting in the upper directory of the source and using the configure/make/make install litany, would be enough. Makefiles do the magic. They will have the calls to gcc, with the different options, etc, so you don't need to call gcc manually for each file. 

Then I'm pretty sure that, by looking at source code and Makefiles, you'll find many, many clues. You'll have to dwell there for some time, I'm afraid, but again, quite probably that will be the fastest way to get the needed skills. The knowledge so achieved will be useful no matter which distro you are using, right now or in the future, debian-based, redhat-based, slackware, whatever.

After that, I think you could easily go with that knowledge to Eclipse and have a successful, quite fast transition to work with that IDE, should you still think you need it.

You are going to embark yourself into a quite interesting experience. I wish you good luck and lots of fun.

[xvr]

It is 28 of .c and .h files. I wanted to put breakpoints to trace through the math which is meandering  over at least 3 of the .c
And the headers are all mixed up hard for me to follow I think due to multiple versions over 10 years.
Can all that be done in eclipse and gdb?

Yes, absolutely.

Eclipse is an IDE that use gdb for debugging, so you need to deal with gdb only.

First of all you need to build project. What build system is used in it? Can you see any of these files at top level?

Makefile
configure
CMakeLists.txt

[mag_therm]

Thanks for the encouraging replies.

The software project I did was actually the largest  single project of my whole engineering career,
It is a numerical simulation of electro thermal process.
Took more than 5 years for me to develop, about 350 k lines of code and uses multi core parallel processing.
Trips around the world to various steel processing plants to measure and verify the accuracy.

At the start I intended multi platform and was going to use C.

Due to deployment and maintenance world wide and expected life of 20 years,
I looked at various "friendly" IDE and eventually selected Xojo.
There were to be no dependencies and the whole application was to be in user's space.
I used a pro programmers initially to do the GUIs and the sqlite.
And client's engineers assisted with scada/plc interfaces

Fairly early in the project, having trouble, we decided to abandon multi platform and just use linux.
We were fortunate that the original code and graphics continued to function over many Xojo updates from 2008 to 2019.
 But in 2019 XoJo did major changes to graphics and so we also had to do an update taking a few months

I suppose all that total experience is how I came to dislike dependencies and to rely on IDE !

And yes, I am partly doing this HackRF1 firmware job to try to keep up to date, as well as hopefully improving the FOSS firmware.

[Siwastaja]

Because their QA team tested it with this version of Python. And their support teams are ready to assist with this version of Python.

Yes. Using Python in the first place is usually a mistake, although no one got fired by using Python, so I guess it's used because it's popular. The problem with Python that fanboys fail to accept but which in actual reality hits nearly every project is that each version of Python breaks compatibility both backwards and forwards (you might not always notice if the project is very simple and thus not using many of the features, but something breaks every freaking time). It's not just Python2 vs. Python3, but really a moving target. You either target one exact Python version and require user to have that exact version, or you maintain your code all day long to match the newest version, and require user to keep updating their Python to the newest release as well. Or you use something as horrible as docker to distribute your software plus the correct Python version.

PSA to all who want to develop software:
 * Do not use bullcrap like Python or docker
 * Avoid dynamic linking, link statically if possible, and therefore:
 * Avoid library dependencies like plague;
 * Only use libraries to do very complex things, like OpenGL for 3D graphics

The danger of "just using a library" to do something as simple as formatting and sending a certain packet was demonstrated on the xz backdoor disaster.

[shapirus]

PSA to all who want to develop software:
 * Do not use bullcrap like Python or docker

What's wrong with docker? It actually provides a way to create an abstraction layer to decouple the software in question from the host OS, thus helping to solve dependency issues, not to mention runtime context isolation.

* Avoid dynamic linking, link statically if possible, and therefore:
 * Avoid library dependencies like plague;

How does the latter follow from the former? If you link statically, you don't care about dependencies.

By using libraries, you avoid reinventing the wheel and avoid making dangerous mistakes. That's what the libraries exist for in the first place. Yes, there are potential security risks, like the xz case. Such cases are rare. By creating your own implementations of what would otherwise be provided by libraries you waste resources and create potential vulnerabilities on your own.