Pets are strange. They are animals that have been bred in such a way that they are entirely or almost entirely dependent on the existence of humans. We keep them, spend extra money on food for them, and various other things that are out of the way. In exchange we get varying degrees of affection ranging from large amounts in the case of many mammals, to little to no affection in the case of many rodents, reptiles and fish.
Take my lizard as an example. I love having her around, it's great. On the other hand, I have to spend a little extra every week because of her. She is incapable of showing me affection, the closest she gets being that she doesn't shy away from me most of the time and is wiling to sit still resting on my fist. She doesn't smell too good (as many pets are) and her terrarium is a hassle to clean up.
Now take fish for example. They are quite incapable of affection. They lack the memory for it. That, and well, they're fish. It's even more of a hassle to clean an aquarium than a terrarium and they do little other than gobble up the bits we throw to them. Fish are an example of a pet whose soul appeal is aesthetic value.
I'm not trying to say that pets are pointless and we shouldn't have them. I love having pets. They are strange, though... as a concept.