Yes, thats possible...make sure the bandit guards don't see you in the first part so you can enter the gates without killing them, then run through abandonned road and open the chests without paying attention to the bandits. After that you only need to fight twinblade and you don't need to kill him.
You can keep trying to sneak past the bandits by loading the autosave if they see you...
So: it's definetely possible, the only hard part is sneaking past the first bandits.
By the way.. you could have also been doing the traders rescue mission.