You might be petting your cat all wrong, and they put up with it because you feed them. No, really. Cats will learn to put up with sub-optimal petting because it's better than no petting at all. As a general rule, cats do not like being stroked on their lower back or their bellies. They much prefer that you rub their ears, face, and neck. But then again, all cats are different, so yours might want it another way.
If you want to give your cat the most satisfying pets, you might want to read a guide from Science Focus. They have a list of body language signals that your cat may be somewhat annoyed at the way you pet him. Try touching him in a new way, and watch for signals of real satisfaction. They even have a procedure for assessing your cat's individual preferences and giving him the affection that he really wants.
(Image credit: LiCheng Shih)