i am guessing there is an earl to go with the castle

Not necessarily. Earls, called counts on the continent, may have castles, but not all castles have earls. Baronets, barons, viscount, earls, dukes, princes, and kings can have castles, too. The wife of an earl is called a countess (viz. Ada Countess Lovelace), while in Scotland earls are called thanes (thegns). Earls are also associated with shires, called counties. Burke's Peerage is a goodly resource: http://www.burkes-peerage.net/ .