Oooh, look, half a point, and you screwed it all up!
Okay, yes, there is a problem when a small group of people hold the majority of wealth in a society, because then that wealth is much more easily weaponized, to basically enslave the rest. Exactly what you're advocating. Chattel slavery has never been good for anyone, most notably for the slaves. But it also results in a complete degeneration of the civilization itself, as having free labor results in stagnation. And say what you will about the cost of caring for slaves, the biggest cost is in labor. And who do you think does all the actual labor to house, clothe, feed, and otherwise care for slaves?
Never . . . at any point in history . . . has any culture's entire wealth been based on gold. Nor in fact, quite probably, has their wealth been based entirely on anything, even when it was based more strongly on wheat, rice, beer, salt, or wood. Want to know how that really works? One region does have great access to vast stores of gold. Another region has no such access, but has a need or desire for gold. The first region needs other things, because they don't have everything. Say, region one needs wood, for example, to support their gold mine tunnels. Region two is a gold deprived forest of dense wooded trees. The two trade their excess in return for their shortage!
As such, every civilization for millennia has based their wealth on not only what they have much of, but what they are short of. This is what makes trade so important, because a man with far more of things his area readily provides, and none of what it doesn't, is still considered poor. Not just socially. He is actually pretty likely to fail to survive without essentials that aren't there . . . which is why he trades the things he can make, grow, or extract from the ground for things he cannot provide for himself. Not only is now socially wealthy, but also more capable of actually surviving. Hence, any way you define it, he is now affluent.
Beyond that, it's come to the point, yes, that individuals can no longer maintain such a trade system to establish wealth . . . or, in other words, increased chances of survival . . . for an entire civilization. There are just far too many people, required to live in ways that we could not live 5000 years ago. 5000 years ago, by the way, is about the last time any kind of "slavery" could really work. Even that slavery was merely a case where an individual or small group was trusted to handle many aspects of decision making on how larger gathering groups would manage their affairs, particularly in their dealings with other such groups, and smaller tribes. They also managed the distribution within their own group, all in return for the work to provide goods to trade. Technically . . . yes, much of it was slavery, but the leaders usually didn't fare much better than their followers.
Compare that to plantation living in the Southern United States and the Caribbean Islands. Anyone who says the blacks of Haiti screwed themselves by kicking out their white masters hasn't been paying attention to what things were like before, and are still focused on skin color rather than action anyway.
