I have always thought of "breaker" as waves that break. But Darwin evidently uses the word for rocks close to shore.
"The coast to the north of Punta Huantamó is exceedingly rugged and broken, and is fronted by many breakers, on which the sea is eternally roaring."