There is a severe alignment issue in the mines, I think its the little beserkers but it could be any of the mops in there. My first time through I was killing everything in there and the party morale was dropping so very fast. I reloaded and let my followers initiate every battle (skipping my turn) and avoiding killing anything that didn't hit on me first (hit nothing without a red circle under it). At the bedrooms on the second level, Dante was *****ing over and over about attacking good aligned creatures which where attacking us on sight. What does he want me to do!? Unless searching for schematics I just run straight to the end of that particular dungeon. Killing the guy at the end is quite a release after the frustration he causes.
Some solutions:
Fate points aren't just for stealing equipped items on npcs
You can use them to force a good reaction from anyone including followers. As soon as I found dog I upped his neutral alignment to LOVE.
Or you can raise your beauty. You can sink character points in there but there are a lot of items and quests that raise beauty, charisma and reaction. Grab the amulet in the chest in the panarii temple in Tarant and loot the ring off the gnome in Grant's tavern (also in Tarant, use a fate point or kill him out of sight of your followers). Another option is the smoking jacket or sunday dress, ok armor for a mage or thief maybe but not practical for anyone taking on a fire elemental toe to toe.
There is a couple of opportunities to raise your beauty in the town of Stillwater which is coming up soon.
My personal advice though is to ditch the whiny hippies, take followers with a bit of moxy.