We are talking about two passengers here, aren't we? This is the equivalent of saying, "Both of the passengers were, at most, hurt only slightly." So why should the number of the verb change just because we have changed the way we say it?

Alternatively, since they, in fact, weren't hurt, this is a contrary-to-fact construction and we use the subjunctive.