The reason that a 17" pitch prop will not work in your case, is that the engine is not turning enough RPM to start with. The prop sort of like screw in the water--and for each revolution theoretically should go 17" forward. (There are also hydrodynamics, involving foil shapes, and higher pressure on the back side of the blade, and lower pressure on the front. An engine has to have enough RPM to spin the prop and achieve its rated RPM at wide open throttle-and when you have too much pitch, you will lug the motor (as you are doing now--with WOT of 4400 RPM--.
Could you put a 6" diameter and 17" pitch, well yes, and you probably could turn up full RPM, but the "slip" of the prop would be huge. A good prop will give you 10 to 20 % slip. You want a happy medium, where you have less slip and most speed at the specified WOT of 5500 to 6000 RPM. Some people "over prop" displacement boats, to get a little better fuel economy--unfortunately this does not work well in planing speed boats.
Here is the link for propeller slip. You do have to know the gear ratio of your lower unit.
http://www.mercuryracing.com/propellers ... ulator.php
If you have bottom growth, you are a candidate for bottom paint--and if you still get marine growth--then either you, or a hired diver should be cleaning the bottom of the boat. When we raced sail boats we cleaned the bottom every week--even with a good bottom paint. We used scotch green scouring pads, or a towel cloth if there was only slime.
There are two excellent books by David Gerr N.A.--The Nature of Boats--has a little on props, as well as a lot on hull design. The other is Gerr's Propeller handbook, which is excellent.
If you don't have the time to read Gerr's book then there are good explanations at:
http://www.boattest.com/resources/view_ ... ewsid=4226