You weren't navigating badly -- Eastern Avenue is the most direct route to the Inner Harbor, and it does go through some areas where a nonresident might not feel comfortable. The better driving route from MTN is west on Eastern Avenue (US40) to the I-695 spur that leads to I-95, then I-95 south through the Fort McHenry Tunnel (stay right!) and the right-hand exit onto I-395 into the downtown by Camden Yards, then follow the signs to the Inner Harbor. Plenty of helpful signs to guide you along this route, which stays out of residential areas.
The FBO is run by the state, not the city, and they can arrange rental cars. The fuel at the self-serve pump over on the left as you're facing the terminal building from the transient line is significantly cheaper.
If you want public transportation, your only real choice is landing at BWI, and letting Signature Flight Support help you get to the rail station. Of course, fuel is dreadfully expensive at BWI, there's a $11 or so landing fee, and SFS lays on a bunch of parking and handling charges, some of which are waived if you buy something like 17 gallons of overpriced fuel.