Excellent guess, et'! And the photograph is a pretty good one of the osage orange fruit that has that distinctive very rough texture.

Ameme: Do you recall anything about the leaves?