This is an interesting one. The word is the same as the gutter at the edge of a street for carrying off excess water. The primary definition of the intrasitive verb is to flow in channels or rivulets. The first sense as applied to candles is that of breaking through the edge of the cup that can form in a candle that is a little too wide for its wick. The derived meaning of alternately burning brightly and dimly would be from the fact that the wick would burn less brightly as it became more deeply immersed in liquid wax and more brightly as the wax level decreased when the edge was breached.

The word came into Middle English through Old French from the Latin gutta, drop.

http://www.bartleby.com/61/29/G0322900.html