"a silly rule, up with which I shall not put."

These things are not always prepositions, see http://wordsmith.org/board/showthreaded.pl?Cat=&Board=words&Number=12094.

A good example of the difference might be something like:

1) Jack and Jill went up a big hill.

This is a classic example of subject-verb-object (prepositional phrase). It would not be correct to say:

2) *Jack and Jill went a big hill up.

But:

3) Jack and Jill ran up a big bill.

Up is not a prepostion. This is an example of the phrasal verb ran up of which a big bill is the object. We can say:

4) Jack and Jill ran a big bill up.

This structure is common to many of the Germanic languages, see the German separable prefix. The form of these separable prefixes is generally identical to prepositions and may, in fact, predate them, but that's another subject.

The notion of the phrasal verb seems to have been lost to the grammar of the prescriptive, if it doesn't match Latin there's something wrong with it, grammarians. It's my not so humble opinion that the cases in which the thing at the end of the sentence actually is a preposition are a result of the burying of the phrasal verb concept by those prescriptive grammarians and the reaction of native speakers who understood the phrasal verb on a native speaker level (without having any formal rule to quote) but were unable to separate it from the prepositional phrase structure.

And a challenge for anyone who thinks that "up with which I shall not put" is correct: parse that phrase; tell me what is the object of each of those things that you think are prepositions.