I have left all sizes boats up to 62 feet unattended at anchor in the channel Islands. But as noted the wind and swell can come up very rapidly and can be dangerous. As long as you set both anchors well, and are appropriate size for the C Dory you should be safe. I have never had an issue in Scorpion.
I spent one of the worse nights I ever had on a boat (including hurricane force winds at sea) at Coches Prietos which is several coves to the South of Scorpion. A swell started to come over the reef at night, which formed breakers which were 8 to 10 feet high on the beach. The boat would surge back on the bow anchor, and I could look over the stern and see the bottom, then as the wave broke the boat would surge forward--as if between rubber bands...
A friend lost his 30 foot Rawson on the rocks in Forney Cove at the West End of Santa Cruz, during a Santa Ana wind. There is no safe harbor on the North side of Santa Cruz during a Santa Ana wind. Probably a cove like the "Willows" would be safe.
I pulled a famous boat named "Passage Maker" out of the surf at Lady's Harbor. The owners had gone for a hike, and not properly set their anchor. We put a line aboard, using our engine and winches worked her off the beach before she was hard aground.
At Fry's I was anchored next to a friend with similar size and type boat. I had a snubber set on my anchor chain--my friend had it a chain stopper, with no elasticity. A heavy surge came in, and his chain broke. I got a line to him and my anchor held both boats.
As illustrated, it may be difficult to get ashore, or to get from shore to your boat, if one of these conditions comes up suddenly. I would have an inflatable dinghy, and be prepared to take it thru surf if necessary. A kayak would work fine. Definitely worth doing some paddling, diving and fishing if you are so inclined. Lots of places to take the boat for day trips to other coves.
If the weather or surf changes be ready to return to your boat immediately... The upper channel islands are far more dangerous than Catalina. Most of this is because of the heavy winds near Point Conception. I have seen these sudden surges at Catalina also--but they seem rarer and don't seem to last as long. Generally the surges I have experienced have not been prolonged at Sant Cruz.
I have used both Danforth and CQR plow anchors at Santa Cruz. The water is moderately deep, be sure you set the anchor in a sandy spot, and back down to be sure it is set. It is the custom to also set a stern anchor so that the bow is set into the swell.
Before you go, beg borrow or buy a copy of "The Diary of a Sea Captain's Wife, Tales of Santa Cruz Island".by Margaret H. Eaton before you go. It is well worth reading about life on the Island. There is a lot of interesting history on the Island. Much has to do with the Stanton Ranch, and the Gherini family had a sheep operation out of Scorpion. I was fortunate enough to have toured the Island during these ranching operations. It is all open to the public with permits now. Not sure if tours are available, but if possible take a tour to the interior. Some great hikes, but take water, watch out for snakes. Not sure if the wild pigs and foxes are still there, but spotting one of those used to be fun. The fox were quite tame.