In cases like this, the rhythm of the sentence is often the deciding factor. In my ears, your sentence without "of" sounds better.