> As AI begins to be applied to problems like driving that affect the real world instead of just information, I’d add:
> Ethical use of AI should not create negative externalities - negative impacts on people other than those making the choice to use it.
I’d go one step further and say that ethical AI should not create any negative externalities to humans period -- not even those that choose to use it.
An example that can be informative here is different than the traditional AI-driving “trolley problem” (i.e. do nothing and kill five people on the crosswalk, or swerve and kill only the passenger), it relates to a more subtle, quotidien challenge faced in autonomous cars: what happens to backseat driving? How will a car that is driven by an AI treat its passengers ethically and insure that it never does them any harm? Even a minor discomfort because, say, an acceleration pattern made them uncomfortable, is an ethical issue if the human has no recourse or if there is systematic prioritization of other goals over the human’s.
One solution is to make cars sensitive to humans’ feelings while they’re being driven, so that the car’s brain has an awareness that, within the right framework, lets it continuously adjust its behavior to serve its passengers better. Essentially, the aim is to make the car empathic, artificially giving it the type of human intuition that can help avoid unintended outcomes big or small. The more that AI systems in general can be sensitive to our human experience, the better I believe.
It is very hard, pragmatically speaking, to always balance the need for easy/cheap AND safe solutions in the real-world. Clearly defined goals and non-goals as well as intermediate steps for measuring against them at least seem to be a good start, since these provide the framework that lets fast feedback mechanisms guide AI to evolve in a way that’s coupled directly with human interest.
I was forwarded this discussion by a friend who’s familiar with my work and I’ve really been enjoying the posts. I think about ethics and AI a lot, and couldn’t help but want to contribute a few thoughts here. So here it goes, my first post on HN..
My main advice is beware of AI’s surprising creativity and proactively work to insure it stays aligned with human interest.
There was a fascinating crowd-sourced paper published this year that shares anecdotes about unexpected adaptations encountered by researchers working in artificial life and evolutionary computation[1]. These are the sort of stories that can be funny in one light (à la “taught an AI to fish and it figured out how to drain the lake”), and doomsdayish in another (“it drained the lake”).
The authors concluded that there is “potential for perverse outcomes from optimizing reward functions that appear sensible.” That’s researcher for ¯\_(ツ)_/¯ ...as Tad Friend wrote in his excellent piece in the New Yorker on this topic[2].
In other words, humans can’t safeguard AI systems solely by defining what they believe to be sensible reward functions.
Reward functions, regardless of whether they are sensible to humans or not, critically need to be mediated by additional regulatory mechanisms, like hard-set non-goals that aren’t just penalty terms relative to a specific reward function. The best non-goals are unequivocally defined and measurable against intermediates that are produced in the reward function optimization process. When done right, this sort of framework allows maladaptive processes to be detected reliably and effective interventions executed.
Tad makes two other points that I think are worth noting in this discussion:
#1. “It will be much easier and cheaper to build the first A.G.I, than to build the first safe A.G.I.”
#2. “Lacking human intuition, A.G.I. can do us harm in the effort to oblige us”
Given #1, when investing in AI companies, if you aim to be “on the more activist end of the spectrum” you’ll need to spend more money relative to market in order to support ethically responsible AI R&D programs, because they will necessarily be harder and more expensive than the irresponsible ones. Assuming you’re investing in AI companies for their products, and not as pure technology plays, this is simply a reality: the core functionalities needed for your portfolio company to sell product X will always be cheaper to develop than the core functionalities needed for your portfolio company to sell product X within a safe, secure framework.
There will be no point for your firm to have codified principles without also having the fortitude to support your AI companies, financially and otherwise, with development processes that are harder and more expensive precisely because they’re more ethical. Many of these costs are absorbed in getting architectures and system designs right, which serve the product anyways, but big costs also come from running unique tests that would be erroneous if ethics weren’t in consideration.
Before thinking about having investees agree to your ethical principles around AI, it may be good for your firm to think about whether you’re willing to pay more for those principles to be lived up to. If two identical companies pitch you with identical AI products, but one plans to take an extra 6 months and $10M to safeguard their technology before launching, while the other intends to capture 8% of the market in that time, who will you fund?
Point #2 relates closely to “perverse outcomes.” In other words, human harm can be unintentional by AIs. Aside from weaponized-AI that does harm intentionally and raises its own separate ethical dilemmas, daily life AI can do damage in a great number of ways without intending to or even knowing it.
The IEEE Standards Association together with the MIT Media Lab recently launched a global Council on Extended Intelligence[3] which addresses many of these issues. Joichi Ito, Director of the MIT Media Lab and a person on the more activist end of the spectrum himself, stresses that: “Instead of trying to control or design or even understand systems, it is more important to design systems that participate as responsible, aware and robust elements of even more complex systems.”
I’d go one step further and say that ethical AI should not create any negative externalities to humans period -- not even those that choose to use it.
An example that can be informative here is different than the traditional AI-driving “trolley problem” (i.e. do nothing and kill five people on the crosswalk, or swerve and kill only the passenger), it relates to a more subtle, quotidien challenge faced in autonomous cars: what happens to backseat driving? How will a car that is driven by an AI treat its passengers ethically and insure that it never does them any harm? Even a minor discomfort because, say, an acceleration pattern made them uncomfortable, is an ethical issue if the human has no recourse or if there is systematic prioritization of other goals over the human’s.
One solution is to make cars sensitive to humans’ feelings while they’re being driven, so that the car’s brain has an awareness that, within the right framework, lets it continuously adjust its behavior to serve its passengers better. Essentially, the aim is to make the car empathic, artificially giving it the type of human intuition that can help avoid unintended outcomes big or small. The more that AI systems in general can be sensitive to our human experience, the better I believe.
It is very hard, pragmatically speaking, to always balance the need for easy/cheap AND safe solutions in the real-world. Clearly defined goals and non-goals as well as intermediate steps for measuring against them at least seem to be a good start, since these provide the framework that lets fast feedback mechanisms guide AI to evolve in a way that’s coupled directly with human interest.