Like Skin stated, as you find armor and weapons, keep the stuff that has the higher damage or better armor rating. The best weapons are the daedric, with the ones from daedric quests in a league all their own. What I would suggest is to find gear that you like or works best for you, then upgrade it via smithing, and if you can, enchant it. Much more effective in the long run.
To get your enchanting up quick, I would suggest Azura's Star. If you complete the quest and get the Black Star, it works like a reusable black soul gem, which is the best soul gem you can get. Use a weapon that has a soul trap enchantment, and everytime a soul is captured into the star, refill your weapon. You level up enchanting fairly quickly.