...a substantial difference between the two languages. What might that difference be?

Sometimes the difference between a dialect and a language is difficult even for linguists to ascertain. For example. some consider Macedonian to be a separate language, some do not. It has been said that a language is a dialect with an Army. This is what might be termed by some, a half-witticism. There are what are considered separate languages that are more closely related than British and American English. If there are severe political/ethnic differences between two groups that have closely related speech patterns members of those groups may resent folks using the linguistic behaviours of the other group when speaking with them. Whether this is sufficient to call them separate languages on a dispassionate linguistic level *should be irrelevant but may raise hackles anyway.