I tend to create my own restrictions when playing.
For example, if I join the Temple, I won't join either the Imperial Cult or Legion.
If I do Gentleman Jim's Bal Molagmer quests, chances are that I'll join House Hlaalu because... spoiler.. highlight below:
they have a similar view on slavery 