Not so non-detectable.

[Blackraven.]

I have a fighter->thief with a cloak of non-detection, yet there was quite a few cases where he would hide in shadows and still be seen by some creatures (without true sight or leaving shadows) such as demons or vampiric mists. Is this their innate ability? Or is it a bug? Even if it was innate, shouldn't the cloak do something?

Or perhaps it's smell?... reminds me of a certain head troll that told me that he could smell me as I got ready to stab him :S

[Saros]

It is written in their script. You are still non-detectable, invisible, i.e. True Sight and the like will not reveal your presence, but those monsters can see and attack even Invisible or Sanctuaried characters. Some other enemies can do it as well. Almost any 'boss' can see you and target you with spells while you're invisible or hidden. Maybe Irenicus is the exception. But for instance if you play Jaheira's quest, Reviane and Dermin will cast spells at, attack and see an invisible character without any trouble, while the rest of their party will stand in place doing nothing.

Most of the tougher demons can do this as well. The Fell Cats in Yaga-shura's lair can also detect you. There are more enemies with such a script, yet I cannot remember all. Ah, yes, liches have this ability too, so don't rely on improved invisibility+SI: Divination when fighting them.

This is not a bug.

[Blackraven.]

Thanks for the reply.
It doesn't make much sense to me, they shouldn't be able to do that IMO. But I guess it's for the best... the game wouldn't be as fun if I could backstab every 'boss' to death or near death before they even get a chance to fight back. It was still a nasty surprise when I was level drained by a swarm of vampiric mists when I thought I was perfectly safe :|.

[Saros]

Yes, I personally too despise Vampiric Mists. They can go invisible whenever they want, can see your characters even concealed/invisible, have some low-level spells at their disposal and their attacks are lvl-draining, and also are fire, ice and electricity immune. Yet, the best way to protect vs them is using a decent Cleric who Turns Undead all of the time, or a good thief, namely Jan, who detects Invisibility all of the time. The other way is using Hold Undead spell or Holy Smite spell or Skull Trap when all of your characters are protected from Magical Energy.

[Thrifalas]

I recall dragons having the ability to see you as well. Mind Flayers are supposed to have it, but doesn't. There are quite a bunch of monsters who can see through it, unfortenately. No more backstabing heaven as BG1 was.

The creature most noticable for having the ability of seeing through insibility is, however, the Planetar. Summonable. Sweet, sweet summon.