I am reasonably sure "continental" was an all-purpose pejorative based on the proverbial worthlessness of it as currency.

I think "continental breakfast" refers to custom abroad of having charge for meals included in price of staying at European hotel, in contrast the "American plan" under which meals and rooms were billed separately. So a "continental" breakfast was not likely to be very filling.