True that is one way of looking at it. The league I'm in has a guy who has won 3 SBs with Buffalo and another guy has won 4 with the Redskins. The league has been around for 35 seasons we play 3-4 seasons a year. I have also seen guys have a bad season with New England, GB and Denver. It is all about a person game skills more than team you play.
A non CFM league would be the only way to accomplish this. I am currently in a non CFM league . The only downfall to this is that we would have to track our records and stats. It is very time consuming whereas with CFM it becomes more like a fantasy league. The downfall of having Panthers vs Panthers is that the majority of us would play as the Panthers.