J.R.R. Tolkien essentially spent his entire life world building and it shows in the depth of his work. The world he created is an integral part of what he has to say. For many authors, however, it is too often nothing more than stage scenery, a painted backdrop, unreal.
This is one reason why my fantasy novels (so far) are all pretty much set in the same well-realized world. There may be thousands of years between the events, but the rules are the same, the way magic works, the ‘gods’ that interact with humanity, and so on. The continents and seas and climate patterns are there, too, all the physical stuff.
I, too, have spent my life creating this world. It is not the only world my fiction visits, of course. Some of my work is mainstream stuff, firmly set in the world in which you and I live. That, too, requires a certain sort of ‘world building,’ the creation of plausible imaginary spaces within the framework of reality. The little Florida towns of my novels SHAPER and THE MIDDLE OF NOWHERE may not exist but I know where the streets run, where the grocery stores are, etc. That is world building just as much as the creation of a fairy realm might be.
And both have to be believable, to seem real. That is how we provide a solid base from which to launch our flights of fancy.
The world builder must be economist and sociologist, geologist and naturalist. There must be some understanding of how things work and interact, how they all fit together. None of this needs to be told in ones tales; it is primarily back-story, things only the author needs to know. Hint at things, throw this and that in during the course of the narrative, but explain nothing unless absolutely necessary!
This is true in all fiction, not just the speculative sort. We needn’t tell why a building is abandoned but we should have a good idea why, and all its connotations about the neighborhood in which it exists, the economy in which the characters move. Knowing these things is world building.
Then, one may turn to the narrative and the character, become a psychologist. That is an whole other role!