An enormous amount of ink has been spilled (some of it by me) on the trials and tribulations of complying with the National Environmental Policy Act, better known as NEPA.
Funny that the two famous overshoots were both throwaway provisions. NEPA's impact statement and the 401(k) got zero debate while the parts Congress fought over mostly landed as predicted. Maybe the 11% isn't a forecasting failure so much as proof that nobody's reading the boring sections until a lawyer finds one.
I wonder if there's a good way to "normalize" the timescale by observing the law's impact 5 years, 10 years etc. after it's official, but would have to account for various macroeconomic factors at the time. Also would be nice to somehow disaggregate the effect on commerce in different sectors (the effect on the environment or education is hard to estimate, but the ban on lead in gasoline presumably had a large effect).
Might be interesting to see if the AI can identify the most costly provisions of these laws and estimate the savings from removing the most costly parts.
There is a vast literature on this topic and the back on envelope stuff here is not useful. As a start you can look at Bruce Yandle’s book Baptists and Bootleggers, or his article with a similar name. Also very important to differentiate laws and regulations. The machinery of regulation and the use of captive agencies by both industry, MGOs. eNGOs, and lobbyists are a large economic literature already. Finally, the enforcement is very often arbitrary and capricious, or at the least subject to high degrees of politicization as a feature not a a bug.
Great article! I'm curious on how the AI justifies that magnitude of Sarbanes Oxley compliance costs. I'm an accounting academic and I don't consider the matter totally settled.
Further complicating the analysis is the reality that some unpredictable results were less unpredictable than others. The impact of SOX was not quite unexpected for the large cohort of lawyers trained in economic analysis of law.
If you think a government can't encourage housing construction, come to the geographic center of So California, Buena Park, Anaheim, Cypress. There is massive building of condos, apartments and townhouses everywhere. This is going to add to traffic congestion and all the other ills of overpopulation. They call it revitalization, when it's really densification. If living here becomes bad enough, I imagine they should get their desire of lower housing prices, as fed up people leave. How the government can afford to subsidize poor people to enable them to live in these houses, is the question.
You should look at the Automotive laws of the 1960s and 1970s. They are the most accurate and important ones as to safety and environment while also significantly improving the auto industry competitiveness vs Japan then.
FMVSS - Federal Motor Vehicle Safety Act.
CAFE - Corporate Average Fuel Economy
Both were enabled, unknowingly; by the Semiconductor and especially, the CPU
Another potential example of a law with radically unintended consequences?
"In 1993, Washington passed a tax cap to look tough on corporate executive pay. Congress failed to understand basic human incentives. It accidentally supercharged corporate greed and built the modern billionaire class."
- John Thorsson
A key reason we have so many billionaires (plus one recent trillionaire): Congress & President Clinton capped the tax-deductibility of corporate executive pay. So corporations issued stock options and did buybacks.
As Thorsson notes in his post, because of this change, the effective pay gap between corporate execs and workers soared:
"According to the Economic Policy Institute, the CEO-to-worker pay ratio entered the 1990s at 60.4-to-1; by the year 2000, it had skyrocketed to 380.9-to-1."
I wonder if prediction markets can be used to identify potential large unexpected effects of laws before those effects occur. For instance, with some new legislation, one can create prediction market for question such as "Will this law have unanticipated effect on the scale of at least ±9-10 in N years?", and let forecasters to propose their probability estimates. That could certainly be useful to try to anticipate future effects of the law.
There are, of course, problems with this idea. Prediction markets resolution criteria are notoriously hard to formulate, to make them as unambiguous as possible. Here we have somewhat subjective scale to quantify unanticipated law effects. Second, it's not very clear, how many years one have to wait before giving a verdict.
Would, say, 10 years be enough to see if law has large unanticipated effects, in your opinion?
One idea to crudely quantify an effect of particular law or specific provision is to count number of times that provision was cited in court proceedings or court decisions.
I wonder if Robin Hanson @wildideaman proposed something along these lines?
This is a very interesting and innovative application of AI. Thanks for doing it.
Funny that the two famous overshoots were both throwaway provisions. NEPA's impact statement and the 401(k) got zero debate while the parts Congress fought over mostly landed as predicted. Maybe the 11% isn't a forecasting failure so much as proof that nobody's reading the boring sections until a lawyer finds one.
I wonder if there's a good way to "normalize" the timescale by observing the law's impact 5 years, 10 years etc. after it's official, but would have to account for various macroeconomic factors at the time. Also would be nice to somehow disaggregate the effect on commerce in different sectors (the effect on the environment or education is hard to estimate, but the ban on lead in gasoline presumably had a large effect).
I like the normalization idea, if someone wanted to pursue this line of investigation more thoroughly that would be a good strategy to use.
Maria De Benedetto et al have some useful papers on this sort of thing
Might be interesting to see if the AI can identify the most costly provisions of these laws and estimate the savings from removing the most costly parts.
There is a vast literature on this topic and the back on envelope stuff here is not useful. As a start you can look at Bruce Yandle’s book Baptists and Bootleggers, or his article with a similar name. Also very important to differentiate laws and regulations. The machinery of regulation and the use of captive agencies by both industry, MGOs. eNGOs, and lobbyists are a large economic literature already. Finally, the enforcement is very often arbitrary and capricious, or at the least subject to high degrees of politicization as a feature not a a bug.
And thar arbitrariness does nothing to alleviate ambiguity and hesitation.
How did NEPA score?
NEPA wasn't in the sample (since it passed in 1969), but a test run scored it at +8.
Great article! I'm curious on how the AI justifies that magnitude of Sarbanes Oxley compliance costs. I'm an accounting academic and I don't consider the matter totally settled.
Further complicating the analysis is the reality that some unpredictable results were less unpredictable than others. The impact of SOX was not quite unexpected for the large cohort of lawyers trained in economic analysis of law.
Thank you for an insightful article.
Thank you for sharing this. This project provides another comprehensive framework to use AI effectively.
If you think a government can't encourage housing construction, come to the geographic center of So California, Buena Park, Anaheim, Cypress. There is massive building of condos, apartments and townhouses everywhere. This is going to add to traffic congestion and all the other ills of overpopulation. They call it revitalization, when it's really densification. If living here becomes bad enough, I imagine they should get their desire of lower housing prices, as fed up people leave. How the government can afford to subsidize poor people to enable them to live in these houses, is the question.
Yes living house to houses and no backyards is terrible for families.
I grew up around there and enjoyed knot’s berry farm.
Beautiful area but shameful housing projects. Stuff as many people as possible in a square acre and charge them lots. Stuck for ???
You should look at the Automotive laws of the 1960s and 1970s. They are the most accurate and important ones as to safety and environment while also significantly improving the auto industry competitiveness vs Japan then.
FMVSS - Federal Motor Vehicle Safety Act.
CAFE - Corporate Average Fuel Economy
Both were enabled, unknowingly; by the Semiconductor and especially, the CPU
Another potential example of a law with radically unintended consequences?
"In 1993, Washington passed a tax cap to look tough on corporate executive pay. Congress failed to understand basic human incentives. It accidentally supercharged corporate greed and built the modern billionaire class."
- John Thorsson
A key reason we have so many billionaires (plus one recent trillionaire): Congress & President Clinton capped the tax-deductibility of corporate executive pay. So corporations issued stock options and did buybacks.
As Thorsson notes in his post, because of this change, the effective pay gap between corporate execs and workers soared:
"According to the Economic Policy Institute, the CEO-to-worker pay ratio entered the 1990s at 60.4-to-1; by the year 2000, it had skyrocketed to 380.9-to-1."
https://firstprinciplesanalysis.substack.com/p/the-loophole-that-built-the-billionaire
Excellent work, Brian, thanks for doing this!
I wonder if prediction markets can be used to identify potential large unexpected effects of laws before those effects occur. For instance, with some new legislation, one can create prediction market for question such as "Will this law have unanticipated effect on the scale of at least ±9-10 in N years?", and let forecasters to propose their probability estimates. That could certainly be useful to try to anticipate future effects of the law.
There are, of course, problems with this idea. Prediction markets resolution criteria are notoriously hard to formulate, to make them as unambiguous as possible. Here we have somewhat subjective scale to quantify unanticipated law effects. Second, it's not very clear, how many years one have to wait before giving a verdict.
Would, say, 10 years be enough to see if law has large unanticipated effects, in your opinion?
One idea to crudely quantify an effect of particular law or specific provision is to count number of times that provision was cited in court proceedings or court decisions.
I wonder if Robin Hanson @wildideaman proposed something along these lines?
Somethings just deserve to be hated.