If you can't resolve the situation, don't restart or anything drastic like that. Having a keep really isn't a big deal in this game (besides the coolness factor). It only really gives you a few more side quests and a small amount of gold (500 a week is really not much, and the side quests cost you a lot).
"But I also made it clear to [Vladimir Putin] that it's important to think beyond the old days of when we had the concept that if we blew each other up, the world would be safe." -President George W. Bush, Washington, D.C., May 1, 2001