that is what I thought at first as well... however that it not the case. no matter how it is held it always shows upside down.
The phone saves the intended orientation (usually based on its acceleration sensor, or an orientation switch inside most other cameras) in an EXIF-Tag within the metadata of those pictures. Some software used for viewing pictures does respect that, some doesn´t, some have an option to do so. If you now rotate the image to match what you see... this information may stay unaltered. For Windows as OS, i´d recommend using a program like irfanview to do some basic editing (rotating, color correction, cropping), it offers many options when loading or saving a file.