As long as we're not getting things thrown our way and the swearing isn't over the top obnoxious, some heckling is all in good fun to me and should be expected. I just don't want my kids or parents to have to worry about fights or beer "spilled" on them.
There might be a heckler or two, but if you have children and elderly parents, anyone who can handle their alcohol will leave you alone. That being said - like others have mentioned, a later game means more intoxicated people. You should be ok, though. Fans may get a little more rowdy in the 500s, but most likely won't be asshats..... unless, of course, you deserve it.