As with insurance, the issue is not the size of the operation, but the nature of what you're doing. There are any number of ways that people can injure themselves on your houseboat--even if it's not your fault--so you would be wise to speak to an insurance broker.
Your general liability insurance may cover invitees, but not renters. As to whether a business license is required, speak to a local attorney. Localities differ as to whether a license or other registration is required for house rental.
Also check any rules and regulations of your local houseboat owners association (if you have one), as they may have restrictions (or need permissions) before you let renters into the area.