> In the simulation, four AI workers are each controlled by their own reinforcement-learning models. They interact with a two-dimensional world, gathering wood and stone and either trading these resources with others or using them to build houses, which earns them money. The workers have different levels of skill, which leads them to specialize. Lower-skilled workers learn they do better if they gather resources, and higher-skilled ones learn they do better if they buy resources to build houses.
So basically they're tackling one of the hardest macroeconomic problems with a simple and completely abstract microeconomic model which sounds less like cutting edge AI and more like a fusion of iterated prisoner dilemma models and the Edgeworth box.
> the AI’s policy cobbled together aspects of both, applying the highest tax rates to rich and poor
Sounds like they've overfitted to the model's assumptions so much their agents are choosing to not be poor primarily to reduce their tax bill...
The question is whether it's useful to have a model at all, or whether it's better to just reason about it from first principles. If a basic simulation gave you a tool to explore emergent properties, perhaps some theories could be generalised to apply to real economies? Or maybe not, I don't know.
An issue I see here is that while punishing the poor is clever way of reducing poverty, a monetary punishment risks trapping them in a negative spiral. Beating them up (but not so much they can't work) or even killing them might work better.
I hope people using AI for public policy are careful what they wish for.
And yet, as simple as this is, four agents is more than you see in most "complex" DSGE models... This looks to me like they're applying deep RL to agent-based modeling. It does seem pretty silly now, but it could be promising. You don't have to restrict these agents to any neoclassical assumptions and there isn't any reason you couldn't introduce firms as well. The only limit is computational power. Macroeconomics tends to look down on agent-based modeling because it doesn't provide neat, closed form solutions. But I've never bought that the results from DSGE models are anything more than products of all their simplifying assumptions. After all, they seem to be the very definition of overfitedness with absolutely no predictive power. At least they're trying something new here.
The headline is misleading (click-bait). Reading the headline, I imagined they have simulated the entire economy of the world and have used reinforcement learning to find optimal policies, but they only simulated 4 workers and are making generalization to the entire economy.
I highly recommend Jeffrey Winters' book "Oligarchy" to understand tax policy. Though an academic book Oligarchy is very easy to read and surprisingly neutral politically.
Winters basically says that oligarchy is when a particular economic group has asymmetric power in a society, arguing that the main use of the power is wealth defense. He then outlines how this has four major forms:
- Warring oligarchies where warlords fight each other to protect wealth
- Ruling oligarchies similar to Italian city states
- Sultanistic oligarchies were multiple oligarchs align to support a single oligarchical ruler
- Civil oligarchies which are basically like the US and Singapore where oligarchs mostly use tax law and investment strategies for wealth defense.
The main evolution between these oligarchies is how much energy needs to be expended in wealth defense, the earlier stages requires constant violence against both the lower class and competing oligarchs, evolving to more cooperation between oligarchs and finally when the wealth class agree to the civil law so long as it can still defend its wealth.
In the case of civil oligarchy, Winters argues that nearly all of wealth defense and manipulation of the law is basically around tax codes and loop holes that allow wealth defense. The oligarchs in general are fine surrendering their military power (which is expensive anyway) so long as they can still defend their wealth. They agree to adhere to the rule of law, so long as the rule of law doesn't impact their ability to maintain wealth.
The conclusion of course being that we will always have complex tax law because it is required by the ruling oligarchy.
This is why we need to get rid of all deductions/credits/exemptions. If the society wants to incentivize something, the government can pay cash for it and account for it properly.
Those frameworks sound a bit vapid and conspiracizing in taking mundane goals and trying to make them sound sinister as possible. Akin to racists taking immigrants and minorities having 3 or 4 kids on average instead of 2 as "planning long term white genocide".
Whether something preserves a given group's wealth is utterly independent of it being good or bad for everybody else. Rule of law and not having bandits choking trade also help to preserve their wealth. So would a hypothetical vaccine for all diseases. The definition of oligarch is also very non-standard as it would not only mean college educated technically qualify as they use their disproportionate earning to earn more to build and preserve their own wealth but so would teotoller factory workers over their peers as they would spend less wealth by not drinking and with compound interest/average stock marker growth and time could become a shocking fund.
To be frank in absense of other details in the summary the "framework" is so vaucous that he can conclude anything he wants including "the agricultural and food producers have ultimate control of the army, navy, and airforce" or "that dentists are the gatekeepers to being upper class".
"In the simulation, four AI workers are each controlled by their own reinforcement-learning models. They interact with a two-dimensional world, gathering wood and stone and either trading these resources with others or using them to build houses, which earns them money. The workers have different levels of skill, which leads them to specialize."
So I'm sure they can create a fairer tax policy ... as long as our world is equivalent to a simplified version of age of empires.
This is just someone's hobby that they somehow convinced other people was serious work.
[edit: Sarcasm removed. I think the project is exciting. I don't think the stated value of data/ML being fairer than politics is very strong because (a.) politics is reflective of culture and represents a sophisticated method of developing consensus among many different ideas of what fair means, (b.) tax accounting work will just shift to GAN and the end result won't be much different just more complex.]
Exactly. I came to say exactly that. Think a hypothetical of when you're a child and your mom has to decide how many cookies your mom gives you and your brother:
* Should mom give you equal cookies?
* Or should she your brother 2x the cookies because you're 5 and he's 15 and has 2x the caloric needs?
* Or should she give you 10x the cookies because he's had so many cookies in his life and you've had comparatively so few?
* Or should he give you more cookies because you're 1/2 his weight and we want to equalize those.
* Or maybe she should give you less cookies because you're fat for your age and he's not.
* Or maybe she should give you more cookies because you like them more than he does and would get more out of them.
* Or maybe she should give you chips because you're allergic to the chocolate in cookies and it would kill you. But what's the right chips to cookies ratio that is fair? And what if your brother also wants chips too, but we only have so many cookies and so many chips and him eating chips means less for you and he can eat cookies, so why should he get chips? But if you're getting 10 chips, and he's getting 0, how is that fair?
I don't know, that seems to match real-world policies, where there are unemployment aids but they're usually contingent on you being employed some of the time (e.g. runs out if you haven't been employed in X months).
> In economics, the Gini coefficient is a measure of statistical dispersion intended to represent the income inequality or wealth inequality within a nation or any other group of people.
The problem is not that we don't know how to create a fair tax policy. The problem is that the unfair winners under current policy spend their wealth to prevent one from being implemented.
It's a human greed problem; an AI cannot help us with that. This AI's advice world either be ignored, or its algorithms corrupted before it began.
> The problem is not that we don't know how to create a fair tax policy.
I would say this is a big chunk of the problem. people don't actually agree on what "fair" means. depending on who you ask, "fair" might mean anything between "I get to keep most of what I earn" and "everyone gets equal access to resources".
regardless of what "fair" means to you, you still need to understand tax incidence (who actually bears the burden of a particular tax) to devise policies that achieve your goal. tax incidence can be a very controversial topic.
What is a fair tax policy, then? Because the vast majority of tax policy debates I've seen are trying to craft a fair policy. Even folks I disagree with vehemently are generally interested in fairness.
It could for sure. But that assumes it is set up correctly.
And even if it is setup correctly and runs realistic simulations, there are often in real life unforeseen variables like hurricanes, earthquakes, new discoveries, etc.
But how we define what is a fair tax policy is incredibly important. If the goal is not well defined how can an AI reach it?
What about other economic factors. What if a fair tax policy is not complete or possible without some other economic element(s) as a counterpart. What if a fair tax policy could only exist when Universal Basic Income is established.
But let's say that somehow magically all unexpected circumstances are accounted for. And the AI does successfully create a fair tax policy.
There will be those whose stand to loose a large amount under a fair tax policy. They are few, but they are powerful and wield massive influence over government. Would they go along with such a policy? It doesn't seem likely. It seems like even If you get a fair tax policy that is created, there will be people honestly questioning it's validity and people who are against regardless of how valid the result is.
It just seems unlikely to succeed. It may be useful economists might dissect what result it gives hoping to find new insights. But beyond that, I'm not sure how much good would realistically come from it.
Something like "fair" tax policy is impossible because the world isn't fair. It's like trying to find mathematical formula to make 2 x 2 equal 5. Anyone trying to shape tax policy is biased one way or another and is causing detachment from effort and reward which is extremely demotivating. It looks innocent on a spreadsheet, but tell someone who worked hard all their life that now they need to part with half of their salary to fund some vanity projects meanwhile they neighbour gets paid for watching telly all day.
Simulation has been used extensively by economists going back decades. One example is the Dynamic Stochastic General Equilibrium model.
Often the output is determined and easily predicted by the input assumptions and the results you get back aren't surprising given an understanding of those assumptions as well as the process that generates the output. Quite often it's a case of garbage in garbage out. Worse yet is when the results are spun as a finding/discovery in the empirical sense when in reality it's very common for researchers to fudge the inputs in order to get an output that is coherent and aligns with both common sense and extant theory.
Simulation (Monte Carlo) is much more reliable in financial product research since the market does seem to obey some stable distributional dynamics, and the consequences on product pricing are well understood.
The linked blog post doesn't mention anything about modelling the scarcity of land, so I'm a little skeptical about the results. In the modern world, one of the most biggest inequalities and barriers to efficiency is access to land, so not modelling that is a big oversight.
It looks like they also only considered income taxes, and ignored all other taxes (such as resources extraction taxes and land taxes), which is also a big oversight.
> However, finding a tax policy that optimizes equality along with productivity is an unsolved problem.
"A land value tax is generally favored by economists as (unlike other taxes) it does not cause economic inefficiency, and it tends to reduce inequality.
Land value tax has been referred to as "the perfect tax" and the economic efficiency of a land value tax has been known since the eighteenth century.", from https://en.wikipedia.org/wiki/Land_value_tax
This sounds rather similar to a lot of the work being done in macro/public finance right now.
If you look at a model from someone like Mendoza you’ll see that they have several parts of the economy characterized as recursive optimization problems + various constraints.
As others have pointed out, however, it is quite difficult to interpret these model results. The best you can do oftentimes is argue for why your setup makes sense then provide some intuition for what is going on. The problem is that slightly tweaked models are capable of delivering qualitatively divergent results.
I do think that reinforcement learning brings to the table some novel techniques for solving models with larger state spaces, though.
What's amusing is that you can probably do it without any 'fairness' tax at all - if there is sufficient saving.
You just need to ensure everybody can sell their labour to the central bank for a fixed amount and that the central bank can credibly threaten to create alternative production at the low end.
Denis Merigoux used formal methods to find discontinuities in the French tax code. This should be a matter of course, done but the OBM for every proposed tax law change.
I know there's a lot of nuances in this, but I really with the US would head towards computer driven policies (with judiciary oversight). Think of it as Census v2.0. There's so many things that should not have human involvement like district lines (gerrymandering).
I'm not saying I disagree with you, but how would this work? At some point there has to be human decision when making the computer program that runs the policy. What's to keep the creators of the program from programming in their bias? Wouldn't the humans still teach the computer how to draw the district lines in a way that benefits their political party?
If politicians wanted things to be fair they could get rid of gerrymandering right now without the need for a computer program. So why would they be ok with a computer that got rid of gerrymandering for them?
[+] [-] notahacker|5 years ago|reply
So basically they're tackling one of the hardest macroeconomic problems with a simple and completely abstract microeconomic model which sounds less like cutting edge AI and more like a fusion of iterated prisoner dilemma models and the Edgeworth box.
> the AI’s policy cobbled together aspects of both, applying the highest tax rates to rich and poor
Sounds like they've overfitted to the model's assumptions so much their agents are choosing to not be poor primarily to reduce their tax bill...
[+] [-] Jabbles|5 years ago|reply
https://en.wikipedia.org/wiki/CovidSim
The question is whether it's useful to have a model at all, or whether it's better to just reason about it from first principles. If a basic simulation gave you a tool to explore emergent properties, perhaps some theories could be generalised to apply to real economies? Or maybe not, I don't know.
[+] [-] im3w1l|5 years ago|reply
I hope people using AI for public policy are careful what they wish for.
[+] [-] Gimpei|5 years ago|reply
[+] [-] clavalle|5 years ago|reply
I'd be interested in playing with such a simulation where the agents must buy or make food to eat and must have shelter or suffer.
And, also, that the other agents know this and can play their own hand accordingly.
I think this would change where the redistribution scheme would land quite a bit if the idea is to maximize value creation.
[+] [-] Cthulhu_|5 years ago|reply
[+] [-] Serious_Cheese|5 years ago|reply
[+] [-] yowlingcat|5 years ago|reply
[+] [-] monadic2|5 years ago|reply
[+] [-] baron_harkonnen|5 years ago|reply
Winters basically says that oligarchy is when a particular economic group has asymmetric power in a society, arguing that the main use of the power is wealth defense. He then outlines how this has four major forms:
- Warring oligarchies where warlords fight each other to protect wealth
- Ruling oligarchies similar to Italian city states
- Sultanistic oligarchies were multiple oligarchs align to support a single oligarchical ruler
- Civil oligarchies which are basically like the US and Singapore where oligarchs mostly use tax law and investment strategies for wealth defense.
The main evolution between these oligarchies is how much energy needs to be expended in wealth defense, the earlier stages requires constant violence against both the lower class and competing oligarchs, evolving to more cooperation between oligarchs and finally when the wealth class agree to the civil law so long as it can still defend its wealth.
In the case of civil oligarchy, Winters argues that nearly all of wealth defense and manipulation of the law is basically around tax codes and loop holes that allow wealth defense. The oligarchs in general are fine surrendering their military power (which is expensive anyway) so long as they can still defend their wealth. They agree to adhere to the rule of law, so long as the rule of law doesn't impact their ability to maintain wealth.
The conclusion of course being that we will always have complex tax law because it is required by the ruling oligarchy.
[+] [-] lotsofpulp|5 years ago|reply
[+] [-] barbarbar|5 years ago|reply
[+] [-] germinalphrase|5 years ago|reply
[+] [-] Nasrudith|5 years ago|reply
Whether something preserves a given group's wealth is utterly independent of it being good or bad for everybody else. Rule of law and not having bandits choking trade also help to preserve their wealth. So would a hypothetical vaccine for all diseases. The definition of oligarch is also very non-standard as it would not only mean college educated technically qualify as they use their disproportionate earning to earn more to build and preserve their own wealth but so would teotoller factory workers over their peers as they would spend less wealth by not drinking and with compound interest/average stock marker growth and time could become a shocking fund.
To be frank in absense of other details in the summary the "framework" is so vaucous that he can conclude anything he wants including "the agricultural and food producers have ultimate control of the army, navy, and airforce" or "that dentists are the gatekeepers to being upper class".
[+] [-] Verdex|5 years ago|reply
"In the simulation, four AI workers are each controlled by their own reinforcement-learning models. They interact with a two-dimensional world, gathering wood and stone and either trading these resources with others or using them to build houses, which earns them money. The workers have different levels of skill, which leads them to specialize."
So I'm sure they can create a fairer tax policy ... as long as our world is equivalent to a simplified version of age of empires.
This is just someone's hobby that they somehow convinced other people was serious work.
[+] [-] HenryKissinger|5 years ago|reply
Greetin. Correctus? Estorer.
[+] [-] adolph|5 years ago|reply
https://blog.einstein.ai/the-ai-economist/
https://einstein.ai/the-ai-economist
https://github.com/salesforce/ai-economist
[edit: Sarcasm removed. I think the project is exciting. I don't think the stated value of data/ML being fairer than politics is very strong because (a.) politics is reflective of culture and represents a sophisticated method of developing consensus among many different ideas of what fair means, (b.) tax accounting work will just shift to GAN and the end result won't be much different just more complex.]
GAN https://en.wikipedia.org/wiki/Generative_adversarial_network
[+] [-] Proven|5 years ago|reply
[deleted]
[+] [-] blippage|5 years ago|reply
To err is human, but to really foul things up you need a computer.
-- Paul R. Ehrlich
[+] [-] tboyd47|5 years ago|reply
[+] [-] pjmorris|5 years ago|reply
> “It would be amazing to make tax policy less political and more data driven,”
Seems like an ...optimistic... viewpoint. IMO, the model, the data, and the notion of fairness are all political.
> "the result appears counterintuitive and not something that a human might have devised."
Will a legislator be willing to enact a policy that doesn't meet their intuitions?
[+] [-] maerF0x0|5 years ago|reply
Exactly. I came to say exactly that. Think a hypothetical of when you're a child and your mom has to decide how many cookies your mom gives you and your brother:
* Should mom give you equal cookies?
* Or should she your brother 2x the cookies because you're 5 and he's 15 and has 2x the caloric needs?
* Or should she give you 10x the cookies because he's had so many cookies in his life and you've had comparatively so few?
* Or should he give you more cookies because you're 1/2 his weight and we want to equalize those.
* Or maybe she should give you less cookies because you're fat for your age and he's not.
* Or maybe she should give you more cookies because you like them more than he does and would get more out of them.
* Or maybe she should give you chips because you're allergic to the chocolate in cookies and it would kill you. But what's the right chips to cookies ratio that is fair? And what if your brother also wants chips too, but we only have so many cookies and so many chips and him eating chips means less for you and he can eat cookies, so why should he get chips? But if you're getting 10 chips, and he's getting 0, how is that fair?
WTF is "fair"?
[+] [-] remram|5 years ago|reply
[+] [-] LatteLazy|5 years ago|reply
[+] [-] clavalle|5 years ago|reply
There are two times to care about inequality:
When the low-end creates physical suffering and when the high end controls enough they can force the low end into suffering.
But, between those markers there's a /lot/ of room to maneuver.
I really feel like the term inequality is a simple placeholder for a more complex idea that has morphed into it's own, mis-aimed, creature.
[+] [-] at_a_remove|5 years ago|reply
[+] [-] kortex|5 years ago|reply
I'm guessing this is their metric.
https://en.m.wikipedia.org/wiki/Gini_coefficient
[+] [-] driverdan|5 years ago|reply
[+] [-] gilrain|5 years ago|reply
It's a human greed problem; an AI cannot help us with that. This AI's advice world either be ignored, or its algorithms corrupted before it began.
[+] [-] leetcrew|5 years ago|reply
I would say this is a big chunk of the problem. people don't actually agree on what "fair" means. depending on who you ask, "fair" might mean anything between "I get to keep most of what I earn" and "everyone gets equal access to resources".
regardless of what "fair" means to you, you still need to understand tax incidence (who actually bears the burden of a particular tax) to devise policies that achieve your goal. tax incidence can be a very controversial topic.
[+] [-] adrianN|5 years ago|reply
[+] [-] pc86|5 years ago|reply
[+] [-] zadkey|5 years ago|reply
But how we define what is a fair tax policy is incredibly important. If the goal is not well defined how can an AI reach it?
What about other economic factors. What if a fair tax policy is not complete or possible without some other economic element(s) as a counterpart. What if a fair tax policy could only exist when Universal Basic Income is established.
But let's say that somehow magically all unexpected circumstances are accounted for. And the AI does successfully create a fair tax policy.
There will be those whose stand to loose a large amount under a fair tax policy. They are few, but they are powerful and wield massive influence over government. Would they go along with such a policy? It doesn't seem likely. It seems like even If you get a fair tax policy that is created, there will be people honestly questioning it's validity and people who are against regardless of how valid the result is.
It just seems unlikely to succeed. It may be useful economists might dissect what result it gives hoping to find new insights. But beyond that, I'm not sure how much good would realistically come from it.
[+] [-] varispeed|5 years ago|reply
[+] [-] grillermo|5 years ago|reply
[+] [-] unknown|5 years ago|reply
[deleted]
[+] [-] clavalle|5 years ago|reply
[+] [-] hnracer|5 years ago|reply
Often the output is determined and easily predicted by the input assumptions and the results you get back aren't surprising given an understanding of those assumptions as well as the process that generates the output. Quite often it's a case of garbage in garbage out. Worse yet is when the results are spun as a finding/discovery in the empirical sense when in reality it's very common for researchers to fudge the inputs in order to get an output that is coherent and aligns with both common sense and extant theory.
Simulation (Monte Carlo) is much more reliable in financial product research since the market does seem to obey some stable distributional dynamics, and the consequences on product pricing are well understood.
[+] [-] captainzidgel|5 years ago|reply
[+] [-] neilparikh|5 years ago|reply
It looks like they also only considered income taxes, and ignored all other taxes (such as resources extraction taxes and land taxes), which is also a big oversight.
> However, finding a tax policy that optimizes equality along with productivity is an unsolved problem.
"A land value tax is generally favored by economists as (unlike other taxes) it does not cause economic inefficiency, and it tends to reduce inequality.
Land value tax has been referred to as "the perfect tax" and the economic efficiency of a land value tax has been known since the eighteenth century.", from https://en.wikipedia.org/wiki/Land_value_tax
[+] [-] andrew3019|5 years ago|reply
I wonder how fairness was measured.
[+] [-] webel0|5 years ago|reply
If you look at a model from someone like Mendoza you’ll see that they have several parts of the economy characterized as recursive optimization problems + various constraints.
As others have pointed out, however, it is quite difficult to interpret these model results. The best you can do oftentimes is argue for why your setup makes sense then provide some intuition for what is going on. The problem is that slightly tweaked models are capable of delivering qualitatively divergent results.
I do think that reinforcement learning brings to the table some novel techniques for solving models with larger state spaces, though.
[+] [-] neilwilson|5 years ago|reply
You just need to ensure everybody can sell their labour to the central bank for a fixed amount and that the central bank can credibly threaten to create alternative production at the low end.
https://new-wayland.com/blog/look-ma-no-tax-from-zero-to-ful...
[+] [-] pradn|5 years ago|reply
Denis Merigoux used formal methods to find discontinuities in the French tax code. This should be a matter of course, done but the OBM for every proposed tax law change.
https://blog.merigoux.fr/en/2019/12/20/taxes-formal-proofs.h...
[+] [-] jl2718|5 years ago|reply
This somewhat obviously achieves both goals by encouraging productivity at the bottom and applying redistribution from the top.
But... real tax systems are designed to maximize revenue, just like any business.
[+] [-] fizixer|5 years ago|reply
Wow, so now the whole of CSE (computational science and engineering) is AI.
Great, I can call myself AI researcher.
[+] [-] syntaxing|5 years ago|reply
[+] [-] buzzerbetrayed|5 years ago|reply
If politicians wanted things to be fair they could get rid of gerrymandering right now without the need for a computer program. So why would they be ok with a computer that got rid of gerrymandering for them?