It really depends on age.
I took my then-5-year-old son up early in my training (dual) once, and put him in the back seat of the 172N as we flew to another nearby airport and practiced landings. I gave him explicit instructions that he was not to speak while we were approaching for a landing until after we had left the runway. It was fine, but I could see where it could be an issue for another kid. I would have taken him on every dual flight afterward, but my instructor wasn't comfy about the liability.
When I did my Piper checkout (and he was 6), I put him in the back seat and he got his first taste of stalls and steep turns. It was a blast for both of us.
Just yesterday, I flew a cross country with him and his mom, to go check out a Gold Rush town in the Mother Lode. This aircraft had a glass cockpit (Aspen) and he asked a billion questions about it, all enroute like he should.
Now, there are some issues, but nothing serious. He has needed to be told not to argue with the PIC, and not to touch any of the controls without permission. Occasionally, he has to be told not to talk over ATC or get in the way of calls I need to make (e.g., contacting ground). The most serious issue is that he has to be told to drink fluids, especially at high altitude, to avoid altitude sickness (not at all a problem, unless he is dehydrated). He tends to prefer the back seat, because he has trouble seeing over the instrument panel. And back seat visibility in the Cessnas especially is much better than you might think. He gets his very own DC10-30 headset.
So far, he hasn't seen any real limitations. He's been as high as 11,500, through continuous turbulence over a mountain pass, and witnessed me having trouble getting the airplane down at Lake Tahoe (too much airspeed, too tight a pattern).
I think it's entirely worth it. I could see some issues with a toddler -- largely solved by having a second adult in the back seat with the kid.