Over here, the janitor is called the "caretaker" - but this is only for public buildings or blocks of flats (apartments to you USns)
In the (good?) old days, men-servants in big houses were NOT called upon to clean and sweep the rooms. Cleaning boots and the cutlery was the job of a boy, part of his training in domestic srvice, from which lowly position he might seek to rise to footman then valet or butler

However, in these modern times there are plenty of cases over here where men do household cleaning, both in the domestic sphere - especially as contract cleaners - and in the hotel trade. They are still very much in the minority, but.