Maybe take a step back to see the bigger picture as it might look on the controller's radar display. What is the controller doing when he releases an IFR flight from N51?
He is protecting an area of airspace around the airport to provide separation for your departure, and for you to maneuver to join your course (V30), prior to establishing radar contact and switching to radar separation standards.
Does it make any difference to him if you turn directly to join V30 immediately (400' AGL after takeoff) vs. loitering in the vicinity of SBJ as you attempt to maneuver to recross it?
What if another airplane is waiting for you to get out of the way in order to begin his approach into, or depart from, N51? Which departure clears the N51 airspace sooner.
Alternatively, *IF* a course reversal to SBJ was required in order to maintain separation with other airspace or traffic, wouldn't the controller include such instructions with your departure instructions? (That's rhetorical. He would.)