There are too many ways to collect stamps to explain them all. If you have 100 collectors in a room they will probably all tell you a different way. Let me try to suggest some approaches.
First, let me explain a bit about value. Many say that a stamp has no value. This is because it is very expensive to prepare stamps for sale and most of the money pays for preparation by the dealer. But, some stamps have value, and I buy stamps from collectors like you all the time. Stay with the hobby and you will learn value. Until you do, I would limit spending to an amount you would any other hobby you expect no return on. If you collect right, your collection will have a value, but a catalog and some education is needed to collect properly for investing.
The newest craze in stamp collecting is what we call topical collecting. A topic can be a dog, a cat, a boat or anything else (even a picture of the pope or a mountain). Other collectors actually pick a topic of the cancel that is used on a stamp. If the cancel is a circular cancel and it is centered on the stamp it is called a "Sock On The Nose" topical. You name it, people look for it.
Topicals also exist from very small countries that sell stamps with a cancel that are created for stamp collectors. These stamps never go through the mail. These stamps are called CTO's (Canceled to Order) and they have no real stamp value to stamp collectors. But they are fun to collect. Please understand and remember that you should never consider a CTO a canceled collectable stamp.
Zoom in on the country you want. I suggest you buy the biggest collection you can afford to buy to start as buying volume will save you money. You might even be able to get the pages for the country and these are usually priced right as pages have very little value. Then you can add with more material and can start saving up duplicates by country.
I wanted to include a few ideas for people who are not collectors. This is a very relaxing hobby good for many hours of fun. You will never run out of things to do and the cost is very reasonable. It can turn into an investment if you like but it does not have to. Why not join me in the interesting world of stamp collecting? Pick the area. Pick the amount you want to spend. And pick a way to collect. There is no wrong way to be a stamp collector.