I don't know about 4-doors, but my 2-door JK's front seatbelts flap wildly in the wind when the doors are off. This creates both an annoying noise, and it vibrates against my shoulder/chest - even more annoying than the sound.

It's the part of the seatbelt running up along the B-pillar that causes the problem. It flaps so violently that the movement transfers all the way to the portion of the seatbelt resting against your shoulder/chest. The solution is to simply add some weight to it to dampen the flapping and reduce the frequency at which it wants to flap.

For a few bucks each, I picked up two of these seatbelt comfort pads at walmart:



They were in the automotive accessories section with the seat covers. There were a few colors available, but I chose to go with the stealth of black.

Just wrap around the seatbelt and attach the velcro to itself:



I found that if you actually attached the velcro strips to each other, it would be VERY loose on the seatbelt (looked sloppy). I wrapped it a bit tighter, and the "hooks" half of the velcro sticks to the fuzzy material of the pad well enough. Leave it loose enough that the pad will always sit down at the bottom. if you wrap it too tight, the pad rights up to the top when you put your seatbelt on.

Several rides at varying speeds confirms that the flapping sounds and feelings are completely gone. Looking at the seatbelt while driving, I see that it still does flap back and forth in the wind, but MUCH more slowly and less forcefully. No sound, and the movement doesn't transfer to the portion of the seatbelt on your shoulder/chest.