Well, how about a quartering wind? Does that not "push" the airplane? Flying sideways is fun!
Only if you correct for it. If you do nothing, you'll move along with the wind.
Let's stop with the push and just say it exerts a force which affects the airplane's path and/or speed relative to the ground.
But that's my whole point - when you're moving along with the wind, it doesn't exert a force on the plane! It only does that if there is relative movement between the plane and the wind. For example:
* the plane is on the ground so it can't move with the wind - the wind exerts a force on the plane
* the plane is in the air but the pilot turns into the wind to correct - the wind exerts a force on the plane
In either of these cases, the plane can "feel" wind. But when you're not correcting for a crosswind, or when you have a tailwind, the plane cannot "feel" it. In other words, the air pressure is the same on the upwind and downwind sides of the plane.