Paddy,
I have both a Whaler and a C-Dory. The Whaler is a 1996 20 Dauntless which is a dual console/bowrider with a 200hp Yamaha SWSII. The C-Dory is a 2008 22 Cruiser with Honda 90 EFI. We bought both new.
The first thing is that they are very different boats. The Whaler tops out in the mid 50 mph range, cruises in the 30's, and burns 12+ gallons per hour. It is also rather noisy, windy, and there is no weather protection besides annoying canvas. The boat is built very well like all Whalers.
Overall I find the C-Dory much more comfortable. As a point of reference, we put 130 hours on the C-Dory this past summer and about 12 on the Whaler. A lot of the use of the C-Dory is going to and from a cabin and even though it generally takes 30-45 minutes longer in the C-Dory, the trip is much more pleasant. The C-Dory is fairly quiet with the aft door closed, there is no wind, if it's cold the heater is on, and if it's hot and sunny everyone is protected from it inside. I find the ride to generally be better on the C-Dory, with the exception of short and steep head seas. However, the impact is much less jarring in the C-Dory because of the speed involved. Unlike the Whaler which wants to be run at 6 mph or 25 mph, the C-Dory can be comfortable run at any speed so it is easier to find a comfortable speed.
When you say you are interested in the 24' model, I assume you mean the 25 (or perhaps Tomcat 24?) My understanding is they are a slightly more comfortable boat in a sea as you'd expect due to the larger size and greater weight as compared to the 22. You may also want to look into the Tomcat 25. I sea trialed one and it was fantastic...great ride, stability, and space, but our dock can only handle boats up to 24 feet and the Tomcat was much more expensive and uses more fuel.
I hope this helped...let me know if you have any more questions.