I just want to point out one or two things.
I work on boats as much as I want to (for the most part), and have a team of others who do the same. We don't have captains licenses, but we are able to work on the water, are all excellent boat operators, instructors, etc. We are frequently looking for temporary/seasonal employees who have a high degree of skill and experience operating boats, and often find these people without a captains license. We also find plenty of license holders who haven't a clue, and shouldn't be on the water.
James' points below should be well heeded. Once you hang up a shingle it's pretty hard to run a charter operation casually. It's a ton of work, especially for the captain. Adding to his list of headaches, remember that you will need crew, and your crew will want to work. If you don't keep at it, you will not retain a good crew, and if you don't have a consistent, happy, and capable crew, your work gets exponentially harder.
So it's a catch 22. You need to push it to get and keep a good solid crew that will then make your work easier. Almost every decent charter captain I know who gave it up simply takes folks out when they feel like it, and basically get them to pay for gas and bait. No other $$ changes hands, so it's just some friends going out fishing. The former capt. gets to keep his boat and use it without going broke paying for fuel and bait, and the passengers get a screaming deal, but might have to be flexible enough to wait for a decent weather window. People who do this seem the most content to me.
There is some considerable expense to maintaining a license. Drug testing for one, especially if you are not employed, can cost you. If you are employed by an entity that participates in approved drug tests (usually random DOT tests) you can simply get a letter from your employer. If not that cost falls to you. As a "Jobbiest" you certainly enter the realm of the latter. The requirements for safety gear are more stringent on a charter vessel, which is another cost.
Finally, I think this was mentioned, but simply holding the license can make your life tough if you are involved in an incident at sea, even if you are just out having fun and it was someone else being stupid that caused the incident. You will be held to a higher standard and scrutinized.
There is plenty to be learned in the class, but not necessary to obtain the license. You just need to take the test and pass. I believe a passing grade in many classes counts as a passed test, but not all.
If there are captains you like a lot in your area you might do as well as a deckhand. That can be a fun job, and certainly less stressful than being a captain.
That said, go for it! Nothing like doing what you want to do, and you may be able to do it all just exactly as you want to do it. You won't know until you try, and if you try, at least you'll know! Every situation is different, and if it doesn't end up the way you want it to, you can stop whenever you want! That's the beauty of being the captain!