Heh, you can't cast Master Speed as a level 20 scoundrel, you need at least three jedi levels
Why the armor restriction? One part because Jedi don't use armor, another part to keep the game more interesting by forcing you to choose. Besides, QD-Robes with decent dexterity (mid-20) will give almost as much defense as the best heavy armors.