Edit:
Sorry for the rotated images. Neither my phone, nor my pc, nor Instagram has an issue with that, only this forum here.
This happens, when people rely on "default settings", which is never good to accept, without having a closer look at all the settings manually; the picures are shown as if being rotated already, but in reality they are not!
As a linux user I don't say anything about windows' internal picture-software, which you probably use, as nothing is able to compete with the free XnViewMP-Software 1.0, where you have the option to do anything you like...
https://www.xnview.com/en/XnView: uncheck "rotate pictures concerning EXIF-alignment"
ACDSee: uncheck "rotate JPEG- and TIFF-files automatically based on EXIF-alignment"
I always rotate my pictures myself, which is very easy to do with ACDSee on a windows PC or XnView on a windows and/or linux PC. With XnViewMP you are able to manipulate the settings the way, that it looks like an ACDSee-Clone, which is simply perfect - yes, it takes a while, but let me know, when you need some help to do so...
Cheers!