You should pretty much find all the information you need here Skyrim:No Stone Unturned - UESPWiki Don't click unless you want a spoiler.
I have found all of them except the one in the embassy. I know its in there, but I can't get back in. I've heard rumors of getting back inside with the aid of a tree-climbing horse, but after an hour or so trying to get my horse to climb a tree by the wall, I am left lacking. Anyone know how to make a horse climb a tree?
What I've heard is that the stone from the embassy got moved to the Reeking Cave (the cave you use to exit the embassy) in one of the patches, so you might want to check there first before abusing your poor horse. If you're on the PC, then you can also use the "tcl" console command to turn off clipping and get back in. SWC
SWC is right you need to find the Reeking Cave which is unmarked and difficult to find . The best directions for this is with Wikia, but maybe this will help. Fast travel to the embassy, move directly to the west of the entrance and down the rocks to the road. Then very carefully move along the road keeping an eye on the right side and a crevis in the rock face will appear, which should be Reeking Cave inside next to a dead priest you should find the stone.
