You can purchase several types of horses. In fact, you even get a free Paint Horse just after the start at Weynon Priory. This is the weakest and slowest horse of them all, but do not buy any other horse because they can be expensive when you are starting out. You can get a horse that is better than all of them for free; the Shadowmere. You can get him after completing the Dark Brotherhood quest "The Purification", which is the first quest you receive when you start working for Lachance. Just kill an innocent person (preferably a beggar if you are not part of the Thieves Guild) and sleep so that you can start the Dark Brotherhood quests. After a couple quests you will get Shadowmere. He is very fast and has a sizable amount of HP. Shadowmere is black/purple, wears a skull on his forehead, has a bloody hand print on the sides of his saddle, and red eyes that glow occasionally at night. He also helps you in battle if you fight near him. Best of all, he cannot die or run away. He is considered an important character and will just fall unconscious if defeated, then will rise again in a couple seconds to join the fray. If you happen to lose him, you can go to Fort Farragut and he will be there waiting for you.