Listen, there's definitely an actual structured game lurking within every GTA game. There's all kinds of dialogue and character interactions and missions for you to follow. Some of them are even sorta fun (some of the time). But a lot of it is tedious. Tedious and boring, especially compared to what you COULD be doing instead (blowing up things and getting into high speed chases and literally everything else). Yet there are still people who want to complete every mission and actually work their way through the storyline, no matter how many RC copters you have to navigate or how many trains you have to follow at just the right distance. And frankly, these people are the true nutjobs, not the ones who shoot up civilians for no reason.
Someone at Rare must have infiltrated the Rockstar offices at some point during the development of San Andreas, because from that point on, the games could easily be confused as some kind of Quentin Tarantino version of Banjo-Kazooie. There were packages, tags, oysters, horseshoes, and more to be collected throughout the various regions of the game. It was scaled back to (mostly) just pigeons in GTA IV, but the virtual equivalent to stamp collecting now goes hand in hand with the murderous impulses of everything else in GTA games from now on - and that's not even including all of the cars, properties, achievements, and miscellaneous other things you can collect and categorize.
Hell, it might be more surprising if stamp collecting ISN'T a feature in GTA V.
One of the best things about GTA games is how much detail goes into them. Every nook and cranny has something interesting about it, which makes a lot of the fun of each game just exploring whatever sprawling metropolis you happen to find yourself in (and then covering it in blood and bullets). There's a reason GTA V cost (an estimated) $265 million dollars to make - building the huge world of Los Santos and filling it with detail doesn't come cheap, especially when you can't just beat up pedestrians and collect their ominous glowing, floating money.