I used to fly up to New Brunswick and Nova Scotia frequently. It's been about 4 years now....stuff happened.
flying around Maritime Canada requires an instrument rating or lots of time and a good weather window. Procedures are straight forward. My experience has been that owners are open to allow pilots fly into their private strips. Generally they ask you to sign something to release them from liability and also generally expect a contribution to the mower and maintenance fund.
You'll run into Mandatory Frequency airports in Canada. We used to have something similar in the US but it's a hybrid of tower versus non-tower...so of like an overly enthusiastic Unicom operator at a non towered US airport. They don't give clearances (they can relay ATC clearances if necessary) but they provide traffic updates and relay your position to other aircraft in the area on call up.
A lot of folks will just land at Halifax and rent a car to tour around the city, Peggy's Cove, Lunenburg, etc. You might want to hit Sydney if you plan to do the Cabot Trail loop by car.
PEI is nice too and a Visit to Charlottetown will be your base of ops for the island. Lots of tourists in the summer so plan your lodging accommodations accordingly.
I haven't ever done it but Grand Manan is a port of entry and supposedly a good stopping point for whale watching tours.
Speaking of overwater, if you don't mind a bit of overwater in order to brag about flying to France, you can do St. Pierre et Miquelon. I think it's about 160nm northeast of Sydney over some very cold Atlantic Ocean. You're spitting distance from Newfoundland too.
If you want to feel like you're in France without leaving Canada you might consider Iles de la Madeleine QC, which is only a 50nm overwater flight from PEI. Probably sufficient as a day trip.
Have fun.
