Many people struggle with this part of the game. the moment you build a base, is the moment the raids start.
So when do you build a base?
The best answer is when it becomes more expensive to feed and equip your squad.
If it is a handful of people, it is strongly advised that you just buy a house in town, put a research bench inside, and research there while looting bandits for food money. Depending on the town, you can sell knocked out bandits to the slavers for money, or in the case of the hub, you will have constant battles in the town from pissed off dust bandits. This is great for training.
Once you expand your squad to 8 people, it becomes expensive to feed each one of them, unless they are skeletons. Making a base becomes necessary at this point. However if you have done the town research you will be more than ready to build a base anywhere, even in the fog, which is now a great location.