Unless you are senile or have some other mental disorder, I don't see how your age matters one bit, so long as you can hold a medical.
Sounds like you just ran into a bad flight school, might want to ask the CFI you got along with if he would freelance with you, or find a experienced freelance guy with a schedule that matches yours, that's what I've done on both sides before.
I am coming up on 60 years old, and I am a youthful 60 (I think), but yes...I definitely think age matters. You do simply slow up a little at an older age learning something new, and this isn't "something", not just a few things, it's a LOT to learn. I've been learning my whole life, I was never static, but I am very sure that I could handle lots more "variable inputs" when I was thirty than I can now. I don't freeze up, but can get fatigued way earlier than I used to.
To the OP. I hear you. It's harder for us old farts to adapt, and handle lots of new things. You have way more hours than I, ten times mine. I only have six hours flying time logged. But I did grow up with a father that was an excellent pilot. I don't know if it helps. When we try to learn a new thing like flying, but with our life experience we are caught in two worlds. If you are like me you have learned that your reaction time is a little (or a lot) slower than it used to be. At the same time, we do know more about physics, and adjusting to new circumstances. I have also had a hard time getting started and I have way less hours than you. My first four flights were with four different instructors, and two different (very different) aircraft.
It's all up to you. In contrast to younger students, who may just accept how it is, you and I may need (and know what we need) to have instruction a certain way. I don't know much, but I would way prefer to learn one aircraft, and have one instructor throughout most of my instruction. In my case circumstances made it such that I had several different instructors. I now have one main instructor and think it is necessary because I feel I have to have someone that knows my weaknesses and my strengths, and can help me work on overcoming my weaknesses. Also I need to have a rapport with my instructor. That takes time. To be able to take instruction in an efficient way. I've trained in two vastly different (one was a very light aircraft with stick, the other is a very old Piper warrior that is now "my" plane) aircraft, and am sure that I need to get the basics down in one and only one, and then try other types.
You do have a say in it. You decide. For myself, I need consistency in order to focus on the fundamentals.
I used to shoot pool. I think it is s lot like that in that the most imoritant thing is to have a dependable stroke, so you can adjust it to make more precise shots. If every time you shot the pool cue acted differently you could never adjust because it would be a crab shoot every time. Something has to be consistent in order to focus on the things that are really important.
Good luck to you, and me.