Sometimes we inherit the double letters from the language we got the word from. Words like illegal and immoral we got from Latin where the il and im respectively were from a prefix in- with the n assimilated by the following letter.