Sidney would be closest. With Nexus, going in to Canada, you call the Nexus number, tell them, among other things, what port of entry you will be going to and when (pretty accurately) you will arrive. We usually call when about 15 minutes away from the POE. You then must go to the POE and tie up briefly, if no one comes to board your vessel you can go ahead. Be sure to get a Canadian Clearance Number from Canada Border folks. Post it on the window of your vessel for the duration of your stay in Canada. When you come back to the States you can call the US Nexus number once you are in US waters (wait until your cell phone is not roaming else you'll be making an expensive international call) You will need to tell US CBP your Canadian Clearance number and some other information. I recommend you call the Nexus numbers in advance to get a "BR" number for each Nexus holder and to give them info about you and your vessel. Save that BR (Boater Registration) number as you will use it for each entry from then on. Nexus doesn't do a whole lot going in to Canada but it is much faster when coming home.
From Sidney you can take a public bus to Butchart Gardens or into downtown Victoria. We've done both and it is clean, safe and easy. Or you can go around the Saanich peninsula and moor closer to the gardens. In summer you might find Todd Inlet a bit crowded. You will need a dinghy to get to the rear gate of the gardens. Check their website.