The word is actually used in two entirely different ways.

First, the older sense I think, the idea that instead of there just being our own universe, there could be another one with different laws or configurations somehow attached to it -- or accessible through wormholes -- or whatever, but essentially a different place, comparable in size to "the" (our) universe. And if there are two universes, why not many such "places"? Then the multiverse is the totality of all such -- as a house contains all its rooms, or a solar system all its planets. There might be doorways, stargates, or other such "passages" between one world and another.

Some modern theories allow for the possibility of such disconnected "universes", and perhaps also access between them.

The other sense is rather more technical, and easily confused with the previous, but quite different in implications. It is that there is no such thing as "our" universe, one among many, but rather the explanation for quantum uncertainty is that all possible quantum results are simultaneously real. There is no one universe which "actually" gets observed, compared to those which might have been. David Deutsch is the champion of this, in The Fabric of Reality, but I think it's unfortunate that he uses multiverse to mean all the simultaneous quantum possibilities. In this sense, it is nonsensical to think of connexions "between" universes: the "universes" are those in which you went to France in 1983 and those in which you didn't, and so on for every other possibility. You can't connect two of those: they're either/or.