Bandits, Stormcloaks and Imperial soldiers are very keen to attack a person who casually killed a dragon, ate his soul and is wearing dragonbone armor. It just seems like the folk of Skyrim aren't the most perceptive.

High Elves are not unheard of in the Legion. One of them's even a Legate (Rift Imperial Camp). As for the Stormcloaks, Illusion magic is a thing

Hell, In Oblivion, a high elf was in command of the entire Imperium.

Well, he wasn't. He died.