How is it propped?
Not knowing the redline RPM or the fuel consumption curve on the motor it's hard to say what is going on and if it's good, but go to:
http://www.boat-fuel-economy.com/yamaha ... us-gallons
you will see that you should be using about 4.70gph at 4500rpm, and that would put you at 5+mpg cruising faster than most with a loaded boat.
It sounds a little too good to be true.
If you are burning more fuel than 4.7GPH you may want to consider a new prop.
If you aren't sure how to determine if your prop is the right pitch, the easy way to do it is to set up the boat the way you'll load it 90% of the time and on a nice calm day with as little current as possible, run it up to full throttle and note the RPM. Do not exceed the rated max RPM, but you should get pretty close to it. I think your motor should max at near 6000 rpm, so you will want to be at or near max RPM, but not over.
Depending on how that goes, you will know if you just have the sweetest fast-cruise CDory around, or if you need a different prop.
My guess is that perhaps your boat takes a little longer to get on step, and is working harder than it should to push your boat as fast as it is going, meaning you are generating more "push" than is ideal with the RPM of the engine.
Another way of looking at it is like a bike with a fixed gear. if it's too high you can go fast, but it's too hard to push the pedals to get it going. If it's too low you will be able to get it going really easily, but your legs will be pumping as fast as they can and you will be going painfully slow. I'd guess you are in the former, geared too high, category.
BUT, it may be perfect, and if it is, WOW!!!! Makes me want a new Yamaha!