I try to remember to just say Eastern because I can never remember which we’re in at the moment.I don't see a problem with it as it is. My dad (RIP) always wished we had DST all the time. My great aunt hated DST because "it's messing with God's clock."
But what really irks me is when people say "Standard" when they don't mean it. For instance, setting up a meeting, someone might say, "we'll meet at 3PM, Eastern Standard Time" when we're actually on EDT. I'm always tempted to show up an hour late for such things. I've been known to insist on a clarification "Standard or Daylight?" which generally yields confused grunts.