There are a few things I consider myself a fan of, for which I show my support in the highly uncreative manner of purchasing things related to those shows (to use the shorthand you introduced). Hence my large and growing collection of anime figurines, wall scrolls, soundtracks, and (of course) DVDs. I haven't written fanfic in more than 15 years, and I can't draw, so this is pretty much the only way I can express my love of something. (As an addendum, I also read the Negima! manga and am so enamored with the world and magic system that I have actually considered contacting Kodansha to inquire about licensing costs–which I imagine would be close to five digits–to do a Negima TRPG.)

I enjoy discussing anime (or whatever) with a small, selected group of relatively like-minded people (like you), but even for shows that I really love I avoid the fan communities as if they were fast-acting poison (because, in my experience, they generally are). The non-anime fan community I was once inexorably tied to (the D&D community) created in me a strong dislike of rabid fans collected en masse (and the messageboards of animenewsnetwork.com only work to further my dread of generalized fan communities) to such an extent that I feel actual physical discomfort when presented with these communities (the City of Heroes forums are the same). It's entirely possible that the fan communities of particular shows, as opposed to the more general anime fan communities, are in some cases made up of reasonable, intelligent, and fun-loving people. It's also possible that sewer rats taste like pumpkin pie. In either case, I'll never know. ("Never" used here more for dramatic effect than as an absolute, because with enough prodding maybe I might consider possibly investigating a fan or fandom community, but it would take more convincing than is worth the time or effort.)

Your mention of the Haruhi fandom is interesting to me because I refer to myself as a Haruhiist (to such a nerdy extent that I sometimes only half-jokingly refer to my religion as being Haruhiism, rather than Christianity) although I do not belong to any Haruhi fan community. Oh sure, I proselytize Haruhi, hard, to the point of making my non-anime-watching (but anime friendly) parents watch episodes with me, I enjoy talking about it with my friends or whenever the discussion of anime comes up, and I buy Haruhi merch, but I don't think the fandom would ever consider me a true fan: I can't dance the Hare Hare Yukai dance, I watch the series in chronological order, I think the dub is really well done, and I like Haruhi more than Yuki or Mikuru.

All of which is a build up to bring me to my ultimate point that, in my experience, those who are casual fans of a show (or a game, or a whatever) are actively discouraged from becoming more engaged as fans due to those who are already among the hardcore fans of the show. At least from my experiences both from within a hardcore subset of fans watching new members being driven off (D&D) or from outside trying to find acceptance and commonality within the fan community (everything else), the insular nature of fan communities make most of them actively hostile to new recruits, and thus limits just how large they can grow. Of course, in some very large fan communities (like D&D or Haruhi or Nanoha) the number of fans willing to put up with ill treatment in order to become part of the communities can be very large, but they remain a self-selecting segment of the overall number of people who enjoy the show (or whatever) that will, over time, steadily decrease in number.