I'd say "go to lunch" implied that the whole group was going out to eat lunch. "Go for lunch" would also have the possibility that one person was going to go out and pick up lunch, bringing it back for everyone to eat.