If you look at a topo map you'll notice that coming to Ithaca from any direction is "going down", as would be going to, say, Watkins Glen. The northern ends of the Finger Lakes don't seem to be so drastically cut into the surrounding territory as do the southern ends, so probably you wouldn't hear the phrase used for Geneva.