Linguistically it sounds kinda funny. I might go with "one-fourth as often" but I don't think there's anything wrong with it mathematically. What would you think "four times fewer" than eight would mean?