That's great!
The sudden "huge gains" were just the point when the things you were having to think about doing became automatic, and you no longer had to think about them. It's something along the lines of "muscle memory." Once the proper technique becomes automatic, you don't think about it, which relieves stress (especially in the neck and chest), which in turn both frees energy and allows the whole body to get into the proper rhythm. Until you get to that point, you have to think about what you're doing, and there's stress. That stress holds you back. Once it's gone, all of your energy is directed into powering the body. The mechanics have become as automatic as walking.
When I was a little kid, I used to swim at the YMCA on 9th Street in Brooklyn several times a week. It was really cheap back then: something like $50.00 a year, I think. They had some good instructors there, and I became an excellent swimmer over time. My swimming skills were later honed in the USCG, and then I took advanced lifesaving in college (mainly because it was an easy Health Ed elective for someone who already was a good swimmer).
It only takes a lifeguard or a swimming instructor a few seconds to size up someone's swimming abilities. Most of an average swimmer's problems have to do, as you pointed out, with the head and arms. Most average swimmers don't breathe properly and don't use their arms correctly. Because they don't breathe properly, they can't keep their heads down; and because they can't keep their heads down, their strokes are awkward and inefficient. Their spines are curved up to support their heads, so the whole upper portion of the arc of their stroke is wasted. The thrust is vectored upwards, as if they were trying to swim up out of the water, rather then being vectored forward. That's where the pull buoy comes in. It forces you to use proper upper body technique.
I'm sort of the designated swim instructor for my family, and over the years, I've taught many relatives and friends to swim in the ocean. It's always fun to watch them when they "get it."
-Rich