I hate the fact that there isn't SWD access and it is crippled by by the Arduino system.
As I hinted above, you don't have to use the Arduino system at all. You can perfectly develop using NXP tools directly.
As for SWD: I'm not sure about the Teensy 4.0, but all other versions include an USB bootloader, so you can Flash them directly through USB with a dedicated app that takes a HEX or bin file. Sure you can't debug... but you can Flash it with just anything you like.
Anyway, yeah as I said there are a bit too few IOs broken out to be really useful. I actually made a custom keyboard (HID) out of a Teensy 3.2, and it was rather quick to do, the number of IOs was alright for this application, but barely. I didn't use the Arduino environment whatsoever.
If you're going to want to drive an LCD screen with it and still have enough IOs left for anything else, you're probably going to have to stick to SPI. Not necessarily ideal for an high-res LCD display, probably adequate for a lower res one.
So yeah don't expect to be able to do anything really fancy around this board (unless of course your project needs very few IOs), but as an evaluation platform for this MCU, it's fantastic and very cheap.