This is very common for departing N07 on a nice day, with delay being caused by nearby CDW/MMU IFR traffic. N90 issues the clearance with "hold for release." You read back the clearance and either pre-emptively say, "I can depart VFR," or, when negotiating the release time if they say "call back in 15 minutes and we'll see how it's looking.." that is a spectacular time to offer the VFR departure.
They'll go speak with the on-duty controller, and inevitably say, "ok, depart VFR, stay on the squawk code and call us in the air..."
You do exactly that, operating as a VFR aircraft on an assigned beacon code. Once you're talking to ATC, they'll radar ID you and then when separation allows, they'll say, "climb and maintain [new altitude]," after which you're considered to be IFR. In fact, most controllers will literally say that as part of the climb, too.
N90 is very much used to this as there is a VFR departure procedure at TEB which has IFR aircraft depart under VFR and then convert to IFR when ATC is able to do so. See the procedure here:
http://nbaa1.wpengine.com/wp-conten...ions/airports/teb/Dalton2_TEB_Publication.pdf and notice the note at the bottom of the page which explains that a subsequent altitude assignment constitutes your IFR activation.
7110.65Z has section 4-3-9 "VFR Release of IFR Departure" which covers this procedure, however, it doesn't specifically talk about the pilot receiving a clearance on the ground then activating in the air. It can be read instead as a pilot calling up and requesting a VFR departure, 'priming the pump' to receive the clearance later. In practice, though, it makes a lot more sense for ATC to inform you of the expected clearance, including routing, altitude and squawk on the ground so that you're not starting from scratch with the busy departure controller, and so that you can process any significant route changes on the ground first.