The only reason weapons get "better" as you go, is because you level.
You get access to elven, ebony, daedric etc. Each weapon type being more stronger than the last.
Best 5 tips IMO:
1. Buy a house in whiterun - $5000 (need to do the Jarls quests). Means you can store stuff easily like Dragon Scales/Bones. They weigh heaps even with a follower.
2. Get smithing up as early as you can. Even if it means spending all your money on leather & leather strips. Or walk around killing bears/wolves (this can take a while). Once you smith items, improve them, then sell it. This will not only level smithing fast, but also speech. Later in the game you'll make more money as a result. You don't even need to "level" the perks.
3. Any magical item, disenchant them soon as you can (gets free/easy levels & perks).
4. Get a follower, and give them armour & melee weapons only. Get them to follow you and suddenly clearing mines/quests is 30-40% faster simply by having an extra pair of hands
5. Don't bother with a horse. YES it takes a while to walk to places sometimes, but doing so will aid you MUCH later on in the game when you can Fast Travel to anywhere on the map.