Easiest way to explain it is this:
Park your plane on the ramp. The airport waypoint being either to the right or left of you by a considerable margin, point is, you cant be right on the waypoint. On the ramp as far as you can conviently get is usually fine. If its a one runway airport, the airport waypoint is usually in the center of the runway.
Turn on the GPS, and put it in regular, not OBS or approach mode, dial in the airport waypoint and push enter etc. Observe the left/right needle. Lets say its pointing left, towards the runway.
NOW, turn the aircraft 180 degrees end for end. Observe the left right needle. Unless your GPS is different than the King KLN90B or the Garmin 430 (and it was quite a while ago version). You will observe the left/right needle pointing LEFT away from the runway. Which is reverse sensing..
Try it in OBS mode also.
Hope the above is all correct. Might be a mistake or two in there somewhere. But I know I've seen this, and seen it in flight too. I've also seen at least one handheld VFR GPS that didn't do it. Like I say, I'm no expert.
Of course you can rid the unit of it's reverse sensing by hitting the GOTO sequence again.
Yup -- poor training on how a CDI presents information. You seem to be thinking that a CDI gives you left/right information, but it doesn't. It only gives you cardinal directional information (i.e., N/S, E/W), and you need to transfer that information to your heading indicator to determine whether that direction is to your left or to your right.
For example, let's assume the airplane was originally pointing north, the direction to the first nav point is north, and the course line from the airport reference point (ARP) to the first nav point is east of you. We'll also assume that you've twisted the CDI to put the course (360, or N) at the top of the CDI, as the OBS has no effect on needle positioning in GPS mode unless the GPS is in OBS mode, but it makes orientation a whole lot easier if you do it that way (and that's why most GPS's bug you if you don't set the CDI OBS to match the DTK). Since the courseline is east of you, the GPS will position the needle to the east side of the CDI (the side where the big E is). If you want to know whether that is to your left or to your right, you look at the heading indicator. With the plane pointing north, east is to your right, so the courseline is to your right, and you have to turn East (right) to get to that line. All's well, yes?
Now, you turn the plane around to the south. The big N is now at the bottom of the HI, the big S is at the top, and the big E is now on the left instead of the right. However, since your location with respect to the selected courseline hasn't changed (i.e., the courseline is still east of you), the needle remains pegged to the east side of the CDI (i.e., towards the big E on the OBS ring), which is still on the right side of the CDI. However, that does not mean then courseline is to your right,
and it never did. If you want to know whether the course line is to your left or right, you'll have to take that information ("course line is to my east), look at the heading indicator, and see whether East is to your left or right. As noted above, after turning the plane around, the E on the HI is now to your left, so the course line is to your left.
However, in either case, since the CDI tells you the courseline is to your east, you will have to fly east to get to it
no matter what your current heading is. To learn whether that will require a left or right turn requires you to look at your HI and see which way you have to turn from whatever heading you're currently on to turn in the shortest direction to East. If you're currently headed north, that will be right. If you're currently headed south, that will be left. IOW, the CDI only tells you the cardinal direction you need to fly to get to the courseline, not whether you will have to turn left or right. For that information, you must use the HI to decide whether that desired cardinal direction is to your left or your right.
For more on this subject, I suggest getting a copy of Peter Dogan's "Instrument Flight Training Manual" (widely available on the internet), and reading Chapter 5 - VOR Equipment, Orientation, and Tracking. It has a full discussion of this issue including diagrams.