Don’t ask if artificial intelligence is good or fair, ask how it shifts power(nature.com)
nature.com
Don’t ask if artificial intelligence is good or fair, ask how it shifts power
https://www.nature.com/articles/d41586-020-02003-2
13 comments
The article is low on examples which doesn't lend it much credibility but I can sort of understand where she's coming from because Google's AI has had similar issues in the past. Namely YouTube classifies LGBTQ+ content as not advertiser friendly and therefore doesn't recommend it as much and often demonitizes it.
Would that not be based on the fact that, that content is somehow generating less revenue?
Or you believe there is a more sinister reason?
Or you believe there is a more sinister reason?
Videos with titles with certain words might be flagged and can’t have ads. As I understand it they’re trying to prevent pornographic content from being monetized, and their list of dirty words include neutral LGBTQ+ terms like “gay” and “lesbian”. This has been verified by experiments where people upload the same video with different titles. [1]
So it’s yet another opaque system that might not have a sinister intention, but has a pretty sinister outcome.
1: https://www.vox.com/culture/2019/10/10/20893258/youtube-lgbt...
So it’s yet another opaque system that might not have a sinister intention, but has a pretty sinister outcome.
1: https://www.vox.com/culture/2019/10/10/20893258/youtube-lgbt...
Reminds me of another case of the opposite with Xbox Live where "gay" and "lesbian" were being censored to try to prevent the then prevalent use as a term of abuse.
To be frank to me the algorithim seems to be a scapegoat for society's own fucked up norms and practices. It isn't a moral process and expecting it to act like one isn't reasonable - it would be like demanding a cargo value calculator stop listing concentrated drugs as the most profitable commodity by to transport per mass and volume because addiction ruin lives and overdose ends them. It is terrible but it is accurately describing the status quo.
To be frank to me the algorithim seems to be a scapegoat for society's own fucked up norms and practices. It isn't a moral process and expecting it to act like one isn't reasonable - it would be like demanding a cargo value calculator stop listing concentrated drugs as the most profitable commodity by to transport per mass and volume because addiction ruin lives and overdose ends them. It is terrible but it is accurately describing the status quo.
Off the top of my head, reasons could be:
- LBGTQ populations are small, about 4-7% (usually quoted at 5%) of the US, and are generally clustered in big urban areas. Like, if you're a queer kid in rural TN you're moving or Nashville or NYC otherwise you gonna have bad time. But that also means it's a niche demographic that's in a few areas. That's a big deal if I'm buying ads for Miami or Denver or The Castro, but it's not broad appeal across the country.
- LBGTQ isn't uniform in context and content; not a ton of lesbians are into boy-meets-boy stories, and not a lot of gay men are watching videos about Andrea Dworkin and S.C.U.M.
- Not a lot of crossover appeal -- I know a few straight bourgeoisie white ladies that are all about queer culture, trying hard to be allies and all, but other than that there is little crossover. Even my wokest, Bernie-Sanders loving friends aren't watching LBGTQ vids and related content on the regular. Again, it's niche thing.
- Serious flak from conservatives, be they Christian, Muslim, or Hindu, or a Communist Chinese government that's not especially LBGTQ-friendly. I can post e-sports videos that will garner millions of views, from Pakistan to China to Nigeria, but a queer critique of Harry Potter or League of Legends characters might be flagged or banned in most of those countries. There may be aggressive pushback and demonetization as well via lobbyists or religious groups.
- As another poster said, flagging words like "gay" or "lesbian" as pornographic and blocking or demonetizing them accordingly.
- LBGTQ populations are small, about 4-7% (usually quoted at 5%) of the US, and are generally clustered in big urban areas. Like, if you're a queer kid in rural TN you're moving or Nashville or NYC otherwise you gonna have bad time. But that also means it's a niche demographic that's in a few areas. That's a big deal if I'm buying ads for Miami or Denver or The Castro, but it's not broad appeal across the country.
- LBGTQ isn't uniform in context and content; not a ton of lesbians are into boy-meets-boy stories, and not a lot of gay men are watching videos about Andrea Dworkin and S.C.U.M.
- Not a lot of crossover appeal -- I know a few straight bourgeoisie white ladies that are all about queer culture, trying hard to be allies and all, but other than that there is little crossover. Even my wokest, Bernie-Sanders loving friends aren't watching LBGTQ vids and related content on the regular. Again, it's niche thing.
- Serious flak from conservatives, be they Christian, Muslim, or Hindu, or a Communist Chinese government that's not especially LBGTQ-friendly. I can post e-sports videos that will garner millions of views, from Pakistan to China to Nigeria, but a queer critique of Harry Potter or League of Legends characters might be flagged or banned in most of those countries. There may be aggressive pushback and demonetization as well via lobbyists or religious groups.
- As another poster said, flagging words like "gay" or "lesbian" as pornographic and blocking or demonetizing them accordingly.
Could it be that _advertisers_ do not want their ads shown next to LGBTQ+ content? For the same reason why men don't see ads for high heel shoes and facial creams, and women don't see ads for Cialis. If that's the case, you'll need to take it up with advertisers, not Google.
If you want to make a clearer example, here's one: Google's ad classifiers used to almost always classify you as a man if you frequent engineering websites, much to the chagrin of many female engineers working at Google.
I don't know if that's still the case, as I'm not a woman myself.
If you want to make a clearer example, here's one: Google's ad classifiers used to almost always classify you as a man if you frequent engineering websites, much to the chagrin of many female engineers working at Google.
I don't know if that's still the case, as I'm not a woman myself.
To be totally honest. How much depth can it be in articles that focus on which groups you have sex with?
LGBTQ+ content is not limited to that.
It doesn't make a lot of sense to lump these groups together in my opinion. LGB seems quite different to T.
They are lumped together because society has done so already. They are outside of the norms and are sexual minorities that challenge the old assumptions and frameworks like men always marry women and the only men without testicles are eunuchs. It is true that LGB are orienation while T is gender identity but they face the same issues. In a societal context where they were all essentially "equally normal as cis-straight" there wouldn't be much of an association just separate categories.
This is a world-view article [1]. Plenty of them are political. Let me copy some descriptions from the top few posts there
> Conservation could be part of the toolkit for diplomacy between China and India.
> How environmental racism is fuelling the coronavirus pandemic
> Reducing your own carbon footprint is not as powerful as calling governments and companies to account.
> Remember what science owes to child refugees
[1] https://www.nature.com/nature/articles?type=world-view
> Conservation could be part of the toolkit for diplomacy between China and India.
> How environmental racism is fuelling the coronavirus pandemic
> Reducing your own carbon footprint is not as powerful as calling governments and companies to account.
> Remember what science owes to child refugees
[1] https://www.nature.com/nature/articles?type=world-view
What is environmental racism?
I find this part particularly bothersome:
> the root problem is that power is distributed unevenly
As if that's some kind of sacred unquestionable axiom that doesn't require supporting argument.
> the root problem is that power is distributed unevenly
As if that's some kind of sacred unquestionable axiom that doesn't require supporting argument.
It is a central part of my personal philosophy, so, for me it's pretty much an axiom.
That being said, sure, sometimes you have to explain why you consider something to be an axiom.
But at the same time, if you where to explain your axioms all the time you wouldn't get very far. So, I guess the question here is: Is it reasonable to assume this axiom in this context?
And I would argue that it is reasonable, otherwise it would be an entierly different text, about a very different subject, and for me personally, it would be a less interesting article.
That being said, sure, sometimes you have to explain why you consider something to be an axiom.
But at the same time, if you where to explain your axioms all the time you wouldn't get very far. So, I guess the question here is: Is it reasonable to assume this axiom in this context?
And I would argue that it is reasonable, otherwise it would be an entierly different text, about a very different subject, and for me personally, it would be a less interesting article.
I can see several logical counter examples of power on a basic level that the thesis isn't consistent or sensible. "Equality" would rely upon too many circumstances to be universal or solveable.
Take 7 bigotted against Y of power X each and add an 8th Y person of power X as well. They could be lynched by them trivially. If they had 7X power he could only be matched if the bigots worked in perfect unision to combine. Otherwise he would be the effective leading power.
If we are dealing with counterfactual root solutions why not use a hypothetical world where violence is literally impossible? It would be more self consistent and just as possible.
Take 7 bigotted against Y of power X each and add an 8th Y person of power X as well. They could be lynched by them trivially. If they had 7X power he could only be matched if the bigots worked in perfect unision to combine. Otherwise he would be the effective leading power.
If we are dealing with counterfactual root solutions why not use a hypothetical world where violence is literally impossible? It would be more self consistent and just as possible.
I think it's pretty safe to say this is true, and it always has been since the dawn of time in almost every culture. It certainly has been for the last few thousand years in Western history.
I don't think there is anything controversial or political about the quote in the article, and certainly not about the impetus to consider the moral application of AI.
I think maybe people are a little worried of what's supposed to be a 'culturally secular' journal, turning over to the groups demanding social reforms in other areas (i.e. CNN yesterday indicating that the term 'Master Bedroom' is 'racist' etc.)
The article is fine, it's on point, very fair. The author is within reason to quote her inspiration.
I don't think there is anything controversial or political about the quote in the article, and certainly not about the impetus to consider the moral application of AI.
I think maybe people are a little worried of what's supposed to be a 'culturally secular' journal, turning over to the groups demanding social reforms in other areas (i.e. CNN yesterday indicating that the term 'Master Bedroom' is 'racist' etc.)
The article is fine, it's on point, very fair. The author is within reason to quote her inspiration.
Ah you see but this is the right kind of political posturing, it is based on leftist ideology and managed to quote a feminist and top of it a BLACK feminist.
Good luck trying to posture with the help of a white,right-wing male. Sorry Ted Kaczynski. I liked your manifesto.
The other ironic thing is the author talking about privilege by publishing in Nature and working at Stanford. Sorry Pratyusha, but you are at the belly of the beast, you are as establishment as it comes, no matter your gender or ethnicity.
Good luck trying to posture with the help of a white,right-wing male. Sorry Ted Kaczynski. I liked your manifesto.
The other ironic thing is the author talking about privilege by publishing in Nature and working at Stanford. Sorry Pratyusha, but you are at the belly of the beast, you are as establishment as it comes, no matter your gender or ethnicity.
I think any technology cannot be created fair from roots. As the technology evolves more fairness will be incorporated , as it would have feedbacks from all sorts of people. I feel this is premature accusation.
The very next sentence is:
> Our network emphasizes listening to those who are marginalized and impacted by AI, and advocating for anti-oppressive technologies.
So do you disagree with the idea that power is problematically unevenly distributed, and disagree that listening to the marginalized and being anti-oppression is good?
If not, and you merely disagree with being explicit about the politics of technology, why? The status-quo politics of AI is "how can we make lots and lots of money from this?". Challenging that politics is good, even if you agree with amoral capital accumulation.
> Our network emphasizes listening to those who are marginalized and impacted by AI, and advocating for anti-oppressive technologies.
So do you disagree with the idea that power is problematically unevenly distributed, and disagree that listening to the marginalized and being anti-oppression is good?
If not, and you merely disagree with being explicit about the politics of technology, why? The status-quo politics of AI is "how can we make lots and lots of money from this?". Challenging that politics is good, even if you agree with amoral capital accumulation.
>So do you disagree with the idea that power is problematically unevenly distributed, and disagree that listening to the marginalized and being anti-oppression is good?
None of this is science. Science does not and can not make value judgements; it cannot say that any distribution of anything is "bad", it cannot say that any particular outcome is good or bad. This has been known for over two thousand years, as the https://en.wikipedia.org/wiki/Regress_argument (mentioned by Sextus Empiricus, the father of empiricism). Moral judgements are the realm of politics and philosophy, and there are plenty of journals dedicated to that. Nature is supposed to be dedicated to science.
None of this is science. Science does not and can not make value judgements; it cannot say that any distribution of anything is "bad", it cannot say that any particular outcome is good or bad. This has been known for over two thousand years, as the https://en.wikipedia.org/wiki/Regress_argument (mentioned by Sextus Empiricus, the father of empiricism). Moral judgements are the realm of politics and philosophy, and there are plenty of journals dedicated to that. Nature is supposed to be dedicated to science.
There's plenty of moral judgements in science. Gene-engineering babies is outlawed. 'Do no harm' is one of the basic principles of medical research. Military science is guided by the interests of the state and its (moral) objectives.
Everywhere you go in science you will encounter moral sentiments at the basis of research. It would be totally equally scientific to experiment on babies, yet we don't do it, and listening to scientists and ethicists on these issues is part of doing science. Ethical debate is intertwined with research.
There is in fact no iron law on how to do science at all and maybe reading some Feyerabend is in order. Science is a social process, and keeping ethics out of science in fact constraints discourse about science in a way that has no justification at all. It impoverishes science by supposing that scientists behave like sort of autistic robots outside of society at large, which they of course are not.
Everywhere you go in science you will encounter moral sentiments at the basis of research. It would be totally equally scientific to experiment on babies, yet we don't do it, and listening to scientists and ethicists on these issues is part of doing science. Ethical debate is intertwined with research.
There is in fact no iron law on how to do science at all and maybe reading some Feyerabend is in order. Science is a social process, and keeping ethics out of science in fact constraints discourse about science in a way that has no justification at all. It impoverishes science by supposing that scientists behave like sort of autistic robots outside of society at large, which they of course are not.
Science is inherently normative. It doesn’t exist in a vacuum without norms or values. Science is ultimately in the service of people, if some distribution is harming individuals, it’s bad. That’s the relativistic moral framing of our society.
It's like you are ignoring science is theory driven and theories are created by people. It's like that nonsense paper in nature that says data is so plentiful we can give up on theories and just find correlations. Except that data isn't objectively delivered to scientists from the universe. The choice of how you measure a phenomena is as theory laden as the explanation.
What are the best arguments for the view that AI scientists are a (possibly innocent and sincere) part of a money-hungry dragon, but political activists aren't a (possibly innocent and sincere) part of a power-hungry dragon?
I disagree that this is the status-quo politics. The vast majority of AI research papers don't discuss making lots of money at all.
Just read your Nature article and loved it. I would like to speak with you to see ways we can help. My new book comes out this year and deals SPECIFICALLY on the asymmetry of power caused by AI. Pls send me details on how we can participate in your work: [email protected]
I'm not sure if machine learning researchers have tools to analyze how minutia of their research e.g. proposing new activation function in ANN will impact shifts in power in the future. I personally have no slightest clue how institutions like police across the world operate and what power dynamics are governing them.
IMO academia is really bad at impacting anything mostly because they are couple of steps detached from actual engineering and management that leads to impactful deployments. Not to mention that academia overall is extremely naive - they failed for some 20 years to stand up for themselves when it comes to research publication. Some researchers even manage to loose rights to access their own research without paying big corporations... For anyone interested in raw power playing academia is like stealing a candy from a kid.
The biggest impact academia could have is on education and that's probably where the focus should be concentrated.
IMO academia is really bad at impacting anything mostly because they are couple of steps detached from actual engineering and management that leads to impactful deployments. Not to mention that academia overall is extremely naive - they failed for some 20 years to stand up for themselves when it comes to research publication. Some researchers even manage to loose rights to access their own research without paying big corporations... For anyone interested in raw power playing academia is like stealing a candy from a kid.
The biggest impact academia could have is on education and that's probably where the focus should be concentrated.
Clearly, the goal of this article is to shift power into the hands of the author and her ideologically aligned community.
Hey it is a good thing that a grassroots, poor NGO like Stanford University get more power.
Well, yes. If you don't like those that wield power you must contest it. Anyone that aims to accomplish a particular goal, such as more equitable AI, will need to challenge the ideology of power and seek to impose their own view.
Generally people accomplish goals by presenting intellectual arguments why the goals are good and should be worked towards. Engaging in power struggles to impose one's view by force is certainly an option that exists, but it's generally seen as toxic behavior and I do not think we should encourage it.
Power inequality is rarely solved by “intellectual arguments”. The NSA didn’t have to debate the American public before wiretapping thousands of citizens, the government used its power to hide and subvert, simply became it was the easiest way to collect such data. They didn’t have to ask each and every person if it was okay to use their cell data before doing so, that’s the power dynamic.
If we read the same article, nowhere is there a suggestion of "[imposing] one's view by force".
Your comment seems to suggest otherwise.
Your comment seems to suggest otherwise.
"Impose their own view" comes from the comment I was responding to, and the commenter did not seem to be suggesting an imposition where "those that wield power" would have a choice in the matter.
I don't think you're reading their comment charitably. They're using "impose" in the context of politics being a contest.
Winning a democratic election necessarily "imposes" upon the losers some decisions of the winners, eg. higher taxes. That's not authoritarianism, that's just politics.
Winning a democratic election necessarily "imposes" upon the losers some decisions of the winners, eg. higher taxes. That's not authoritarianism, that's just politics.
>Winning a democratic election necessarily "imposes" upon the losers some decisions of the winners, eg. higher taxes. That's not authoritarianism, that's just politics.
This doesn't change that it's the majority imposing its will on the minority by force.
This doesn't change that it's the majority imposing its will on the minority by force.
This is all true, and it's also true that those in power use power to serve their own interests and the interests of their group. That's why power must be contested. If you're neutral, power is serving you or else you'd be incensed.
Responding to the comment below: Well, then the stakes for you are low enough that you don't care, you feel too powerless to protest, or the decisions are in some sense fair. Very often people are subjected to decisions that are not fair and they should dissent.
No, I fundamentally reject the framing. I don't divide the world into things that serve me and things I'm incensed about. Neutral institutions regularly do things that go against my interests, and I don't get angry or seek to overthrow them every time it happens.
The core arguments, as far as I can tell, are:
>Researchers should listen to, amplify, cite and collaborate with communities that have borne the brunt of surveillance: often women, people who are Black, Indigenous, LGBT+, poor or disabled. Conferences and research institutions should cede prominent time slots, spaces, funding and leadership roles to members of these communities.
And:
>Remarkably little research focuses on serving data subjects. What’s needed are ways for these people to investigate AI, to contest it, to influence it or to even dismantle it. For example, the advocacy group Our Data Bodies is putting forward ways to protect personal data when interacting with US fair-housing and child-protection services.
I'm unable to find any other concrete suggestions or arguments. There are some things like "In addition, discussions of how research shifts power should be required and assessed in grant applications and publications.", but "how research/AI shifts power" seems to be a very fuzzy and ill-defined concept.
People aren't going to agree on what power is and who possesses it and who doesn't and what does or doesn't constitute shifting or non-shifting of power, and to who, and how, and why. Let alone exactly how their research's tiny piece of the puzzle fits into it all.
I think it'd be clearer if this contained specific claims or requested specific legal rights beyond broad terms like "shifting power": like anonymization of data, regulations on how companies and the government can collect and use people's data, how and when law enforcement can use automatic recognition systems, etc. Maybe an auditing organization for companies' and governments' heuristic systems (AI or otherwise), to look for type I errors (YouTube LGBT+ videos as not suitable for minors comes to mind). Maybe some way to determine what technologies could be ripe for abuse.
It's like making your motto "our organization is dedicated to ensuring AI benefits the people, not the elites". Okay... who are the elites, who are the people, how could the elites and people be benefitted or harmed, etc.
>Researchers should listen to, amplify, cite and collaborate with communities that have borne the brunt of surveillance: often women, people who are Black, Indigenous, LGBT+, poor or disabled. Conferences and research institutions should cede prominent time slots, spaces, funding and leadership roles to members of these communities.
And:
>Remarkably little research focuses on serving data subjects. What’s needed are ways for these people to investigate AI, to contest it, to influence it or to even dismantle it. For example, the advocacy group Our Data Bodies is putting forward ways to protect personal data when interacting with US fair-housing and child-protection services.
I'm unable to find any other concrete suggestions or arguments. There are some things like "In addition, discussions of how research shifts power should be required and assessed in grant applications and publications.", but "how research/AI shifts power" seems to be a very fuzzy and ill-defined concept.
People aren't going to agree on what power is and who possesses it and who doesn't and what does or doesn't constitute shifting or non-shifting of power, and to who, and how, and why. Let alone exactly how their research's tiny piece of the puzzle fits into it all.
I think it'd be clearer if this contained specific claims or requested specific legal rights beyond broad terms like "shifting power": like anonymization of data, regulations on how companies and the government can collect and use people's data, how and when law enforcement can use automatic recognition systems, etc. Maybe an auditing organization for companies' and governments' heuristic systems (AI or otherwise), to look for type I errors (YouTube LGBT+ videos as not suitable for minors comes to mind). Maybe some way to determine what technologies could be ripe for abuse.
It's like making your motto "our organization is dedicated to ensuring AI benefits the people, not the elites". Okay... who are the elites, who are the people, how could the elites and people be benefitted or harmed, etc.
Lol. “Communities that have borne the brunt of surveillance” and then a list starting with “women.” Men are surveilled at vastly higher rates than women, and a huge majority of police brutality is against men (specifically, poor and black men). Next up is “Indigenous” which is also laughable, as at least in the US the federal government clearly couldn’t give two shits about what is or isn’t happening to Native American people.
The fraction of people in AI working on problems that need to consider diversity/fairness/etc. is rather small. Yes, those people, these specific applications, should be designed with care, should be overseen properly and in the best of all worlds should be unbiased. However, the recent discussion around this topic is framed like AI research in general is somehow unethical and should be informed more by minority populations. While this is something to strive for, for the sake of having a more just society, the impact of diversity on an AI system that say classifies mitochondria in yeast is not clear to me. I’d argue the majority of problems in AI right now are of this form and not of the form addressed by the author. If you want to push quotes, increase minority representation for occupations, do so. If you want to address specific biases in very specific applications then do that. Don’t use the latter to make an argument for the former though. These are two different issues and while I recognize the author sees a link here I’m not so certain about that. To be clear I think that both are worthy goals. But it’s somewhat disingenuous to make an inherent political argument, people may disagree with, and justify it by overgeneralizing a specific, niche technological ethics question to an entire field.
I don't think you're making much of an effort to describe the variety of AI that are appearing. The thing about AI is it thrives on data and lots of large institutions have data that they use to make decision about people. Fairness questions come in.
Google randomly, I see:
AI Applications: Top 10 Real World Artificial Intelligence Applications
Marketing, Banking, Finance, Agriculture, Health Care, Gaming, Space Exploration, In Autonomous Vehicles.
About half of those seem like situation where fairness questions enter - banking and finance - plausibly (who gets a loan), marketing - plausibly (who gets sold what), health care - plausible (who gets treated, what groups' data is and isn't used for what, etc). The other not so much.
Google randomly, I see:
AI Applications: Top 10 Real World Artificial Intelligence Applications
Marketing, Banking, Finance, Agriculture, Health Care, Gaming, Space Exploration, In Autonomous Vehicles.
About half of those seem like situation where fairness questions enter - banking and finance - plausibly (who gets a loan), marketing - plausibly (who gets sold what), health care - plausible (who gets treated, what groups' data is and isn't used for what, etc). The other not so much.
In industry, yes. I had the impression this article was aimed at academia and research in AI.
Well, academia pretty has to study AI applications.
AI is not a field like physics, which can be roughly separated into theoretical tools and applications of those tools.
AI is creating heuristics, approximations to data that "generalize" while keep how that generalization works vague. Essentially it's a very leaky abstraction so researchers need to be concerned how that leaking happens, what it's implications are.
AI is not a field like physics, which can be roughly separated into theoretical tools and applications of those tools.
AI is creating heuristics, approximations to data that "generalize" while keep how that generalization works vague. Essentially it's a very leaky abstraction so researchers need to be concerned how that leaking happens, what it's implications are.
I happen to work in AI research and what you are saying isn’t true. There is theoretical machine learning and applications of it. They are distinct.
The former is largely task agnostic and deals with fundamental issues such as, how to train networks in an unsupervised way, how do you do hard statistical inference for intractable models in a approximate but well enough manner, how is gradient descent behaving exactly, why does it work, are there better ways for optimizing nns, what about pruning? The list goes on.
On the other hand you have applications of AI to other fields that require their own research. In biology for example you may want to segment cells and their compartments or design better point spread functions for you microscope or classify cell types. These are applied problems.
Again, as stated in my original post, striving for more diversity is a good thing and should and is done. Why make it about AI ethics and bias though when large portions of this field have no contact point with it?
The former is largely task agnostic and deals with fundamental issues such as, how to train networks in an unsupervised way, how do you do hard statistical inference for intractable models in a approximate but well enough manner, how is gradient descent behaving exactly, why does it work, are there better ways for optimizing nns, what about pruning? The list goes on.
On the other hand you have applications of AI to other fields that require their own research. In biology for example you may want to segment cells and their compartments or design better point spread functions for you microscope or classify cell types. These are applied problems.
Again, as stated in my original post, striving for more diversity is a good thing and should and is done. Why make it about AI ethics and bias though when large portions of this field have no contact point with it?
In your research, how do you define the process of "generalizes"?
I understand theoretical research exists but I think it's problematic that theoretical researchers imagine that a kind of "generic" problem exists, even when a variety of test sets exist to
I mean, is SOTA on imagenet or whatever data a theoretical or an applied question? What theoretical research in AI is so theoretical that the question of data sets doesn't appear?
I understand theoretical research exists but I think it's problematic that theoretical researchers imagine that a kind of "generic" problem exists, even when a variety of test sets exist to
I mean, is SOTA on imagenet or whatever data a theoretical or an applied question? What theoretical research in AI is so theoretical that the question of data sets doesn't appear?
Let me be very concrete: There is currently a lot of research being done in so called “contrastive learning”. This is an unsupervised technique in which you train a network to build good representations of its input data. Not by explicitly telling the network, “this is a dog” but by telling it “this image is different from that image and similar to this other one”.
How this works, why this works, coming up with the technique itself, are all data agnostic. All you did so far is write down a function f(x) -> y and a loss L(x,y), with x the input and y the output, specifying your model.
Of course you use a specific dataset to train your model in the end and see if it works. But the model and the technique itself are not grounded in any specific dataset and thus nothing in this model perpetuates bias.
Now usually the next step for you as a researcher is to evaluate the performance of your model on the test set. Lets take image net. Now there are 3 situations.
A) Your Train set is biased & your test set is biased in the same way.
B) Your train set is biased & your test set is not.
C) Your train set is unbiased & your test set is biased.
With biased I mean any kind of data issue such as discussed in this article, e.g. no women in the class “doctor”.
In situations B) & C) your model wont work well so you actually have an incentive to fix your data. This will happen to you in production if you train your tracker only on white people say.
Situation A) is likely what’s happening with imagenet and other benchmark datasets. In this case your model learns an incomplete representation of the class “doctor” and learns the spurious correlation that all doctors are men. This will work on the test set because it’s equally biased.
You go, get good test results and publish a paper about it, unaware of the inherent dataset biases. (You could have done all this also on MNIST or a thousand other datasets that do not have any issues with societal biases because they are from a totally different domain, but that’s another point).
In this entire process of coming up with the model, training and evaluating it, there is no point at which the researcher has any interest, or benefits from, working with biased datasets. Furthermore, besides potentially overestimating the accuracy of your model, there is nothing in here that would hurt society or further perpetuate biases. That is because models are generally not designed to work on a specific dataset.
Again, this is a different story when you use your model in production. In this case you are in situation B) or C) and here now lies the crux. If you can make money from this bias or maybe it perpetuates your own biases well you might keep it like that. This should be fixed. Here now is a real argument for why there should be diverse populations working on AI systems that are used in industry.
Of course having diverse populations in science is also our goal. But not to fix our datasets but to do better research.
How this works, why this works, coming up with the technique itself, are all data agnostic. All you did so far is write down a function f(x) -> y and a loss L(x,y), with x the input and y the output, specifying your model.
Of course you use a specific dataset to train your model in the end and see if it works. But the model and the technique itself are not grounded in any specific dataset and thus nothing in this model perpetuates bias.
Now usually the next step for you as a researcher is to evaluate the performance of your model on the test set. Lets take image net. Now there are 3 situations.
A) Your Train set is biased & your test set is biased in the same way.
B) Your train set is biased & your test set is not.
C) Your train set is unbiased & your test set is biased.
With biased I mean any kind of data issue such as discussed in this article, e.g. no women in the class “doctor”.
In situations B) & C) your model wont work well so you actually have an incentive to fix your data. This will happen to you in production if you train your tracker only on white people say.
Situation A) is likely what’s happening with imagenet and other benchmark datasets. In this case your model learns an incomplete representation of the class “doctor” and learns the spurious correlation that all doctors are men. This will work on the test set because it’s equally biased.
You go, get good test results and publish a paper about it, unaware of the inherent dataset biases. (You could have done all this also on MNIST or a thousand other datasets that do not have any issues with societal biases because they are from a totally different domain, but that’s another point).
In this entire process of coming up with the model, training and evaluating it, there is no point at which the researcher has any interest, or benefits from, working with biased datasets. Furthermore, besides potentially overestimating the accuracy of your model, there is nothing in here that would hurt society or further perpetuate biases. That is because models are generally not designed to work on a specific dataset.
Again, this is a different story when you use your model in production. In this case you are in situation B) or C) and here now lies the crux. If you can make money from this bias or maybe it perpetuates your own biases well you might keep it like that. This should be fixed. Here now is a real argument for why there should be diverse populations working on AI systems that are used in industry.
Of course having diverse populations in science is also our goal. But not to fix our datasets but to do better research.
>In my view, those who work in AI need to elevate those who have been excluded from shaping it, and doing so will require them to restrict relationships with powerful institutions that benefit from monitoring people.
I am not going to choose who I work with based on their race/sexuality/gender.
>Researchers should listen to, amplify, cite and collaborate with communities that have borne the brunt of surveillance: often women, people who are Black, Indigenous, LGBT+, poor or disabled.
Right, we know, only privileged straight white men escape the clutches of surveillance.
All these ideologues are doing is setting the stage for the marginalization of whites, a global minority and soon to be minority majority in the US. I can't believe this kind of casual discrimination has not only been normalized, but is openly and explicitly encouraged.
This is not ok.
Edit: you know, the same way that it takes generations to build wealth, it takes generations to build knowledge, especially institutional knowledge. If we just hand over a chunk of our industries to people because of their identities, by definition we will be even less of a meritocracy - and that's bad for all of society.
I am not going to choose who I work with based on their race/sexuality/gender.
>Researchers should listen to, amplify, cite and collaborate with communities that have borne the brunt of surveillance: often women, people who are Black, Indigenous, LGBT+, poor or disabled.
Right, we know, only privileged straight white men escape the clutches of surveillance.
All these ideologues are doing is setting the stage for the marginalization of whites, a global minority and soon to be minority majority in the US. I can't believe this kind of casual discrimination has not only been normalized, but is openly and explicitly encouraged.
This is not ok.
Edit: you know, the same way that it takes generations to build wealth, it takes generations to build knowledge, especially institutional knowledge. If we just hand over a chunk of our industries to people because of their identities, by definition we will be even less of a meritocracy - and that's bad for all of society.
What is "soon"?
(https://en.wikipedia.org/wiki/Demographics_of_the_United_Sta... suggests "A report by the U.S. Census Bureau projects a decrease in the ratio of Whites between 2010 and 2050, from 79.5% to 74.0%")
https://en.wikipedia.org/wiki/The_Rising_Tide_of_Color_Again... was almost exactly a century ago. Might be worth a read before we go stampede cattle through the Vatican.
(https://en.wikipedia.org/wiki/Demographics_of_the_United_Sta... suggests "A report by the U.S. Census Bureau projects a decrease in the ratio of Whites between 2010 and 2050, from 79.5% to 74.0%")
https://en.wikipedia.org/wiki/The_Rising_Tide_of_Color_Again... was almost exactly a century ago. Might be worth a read before we go stampede cattle through the Vatican.
“Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.” - Dune
That is a pretty ironic given the sentiment is given from actual slavers who justify it over their ancestors not fighting the machines.
Since when is Nature an outlet for political propaganda and racial discrimination?
Seriously, what a useless article.
It is concerning seeing Nature viewing everything from such a postmodernist perspective of power dynamics.
The best thing you can do with powerful technology is to give it away for free. Then everyone is on even footing and you don’t get an abusive power imbalance.
> Then everyone is on even footing and you don’t get an abusive power imbalance.
This doesn't hold up to even a first test using a real-world example.
Consider making guns free. Even if everyone owned an identical gun with identical ammo (an asburdly artificial situation to begin with), not everyone can use a gun equally effectively.
Apply it now to crowds. If everyone owns the same gun, it takes far fewer people to band together and take physical control of a certain area. Even an individual with that same gun is effectively powerless again even two others: to kill one is to be certainly killed by the other.
Now swap the guns with fighter jets, zero-day exploits, bioweapons, fame, religious leadership, or any other form of power.
There is seemingly no scenario where your axiom seems to hold up, even in contrived circumstances.
This doesn't hold up to even a first test using a real-world example.
Consider making guns free. Even if everyone owned an identical gun with identical ammo (an asburdly artificial situation to begin with), not everyone can use a gun equally effectively.
Apply it now to crowds. If everyone owns the same gun, it takes far fewer people to band together and take physical control of a certain area. Even an individual with that same gun is effectively powerless again even two others: to kill one is to be certainly killed by the other.
Now swap the guns with fighter jets, zero-day exploits, bioweapons, fame, religious leadership, or any other form of power.
There is seemingly no scenario where your axiom seems to hold up, even in contrived circumstances.
> Now swap the guns with...
muscles? upper body strength?
that free gun world sounds pretty good.
> Apply it now to crowds. If everyone owns the same gun, it takes far fewer people to band together and take physical control of a certain area.
that in no way follows.
muscles? upper body strength?
that free gun world sounds pretty good.
> Apply it now to crowds. If everyone owns the same gun, it takes far fewer people to band together and take physical control of a certain area.
that in no way follows.
> muscles? upper body strength?
The original post was about technology.
> that in no way follows.
Scenario A: No one has a weapon. Five people take control of a grocery store. How many people can take back the store without anyone dying? The answer might be as low as five. Certainly 10-15 people could do it.
Scenario B: Everyone has an identical gun. Five people take control of a grocery store. Same question.
The answer is that there is no number of people high enough to plausibly reduce the risk of death to zero. Someone with a gun can kill much faster and at greater distance.
Long story short: technology amplifies power. Sufficiently amplified power cannot be countered by equal power quickly enough to nullify the danger.
The original post was about technology.
> that in no way follows.
Scenario A: No one has a weapon. Five people take control of a grocery store. How many people can take back the store without anyone dying? The answer might be as low as five. Certainly 10-15 people could do it.
Scenario B: Everyone has an identical gun. Five people take control of a grocery store. Same question.
The answer is that there is no number of people high enough to plausibly reduce the risk of death to zero. Someone with a gun can kill much faster and at greater distance.
Long story short: technology amplifies power. Sufficiently amplified power cannot be countered by equal power quickly enough to nullify the danger.
Nuclear proliferation is a counter example. Its danger goes up the more states that have the tech.
I'm opposed to nuclear weapons, but: the principle of nuclear non-proliferation - or, in other words, the oligopolization of nuclear strike capability - is more of a subjective and self-serving value held by the US (and some other nuclear powers) than a universal objective truth. If, hypothetically, the US lacked nuclear weapons that other countries had, the acquisition of nuclear capability by a hitherto non-nuclear USA would suddenly be seen as very desirable, and there wouldn't be all this talk about the dangers of proliferation.
Nuclear weapon proliferation is actually two risks fighting each other for total risk curves fighting each other - conventional war prevention and irrational actor risk. More proliferation prevents conventional wars. Pakistan and India have limited themselves to skirmishes at worst. However the more proliferation the greater the risk of a suicidal actor who doesn't care about or seeks mutually assued destruction.
If you want world peace betweeen nations at any cost give every nation ICBMs and nuclear warheads sufficient to break through enough of everyone's defenses - it will last unless and until thermonuclear war starts.
If you want world peace betweeen nations at any cost give every nation ICBMs and nuclear warheads sufficient to break through enough of everyone's defenses - it will last unless and until thermonuclear war starts.
North Korea has nuclear weapons and they seem to be the looniest state around (Russia, America, and China also have nuclear weapons); I think this horse has left the stable.
Take a look at the trajectories North Korea has used in their missile tests. Their leadership may well be as loony as claimed, but whoever is picking parabolas knows how to speak softly and loft a high apogee.
(similarly, no matter how loony their current CINC may be, whoever in the USAF redeployed B-52s from Andersen AFB to Minot AFB also understands aposematism)
(similarly, no matter how loony their current CINC may be, whoever in the USAF redeployed B-52s from Andersen AFB to Minot AFB also understands aposematism)
I saw a comment once pointing out that once a country has nuclear weapons, it tends not to be threatened by others. At the level of individual countries, the danger goes down.
More dangerous, less power imbalance.
That works well with software, but it looks like AI is trending towards requiring some pretty hefty hardware, especially for training.
It will be interesting to see if giving away the software is enough.
It will be interesting to see if giving away the software is enough.
Marxian analysis is the only analysis now, apparently.
It is going to get worst. If you think carefully and quoting Feynman "There is plenty of room at the bottom".
Do you want examples?
A Simon Bolivar statue was vandalized in Florida, now Bolivar was no saint but he is a South-American hero he died in 1830 and now he has been tarnished just by being a "White Slaver"(He was not). Well, let's see you can come now after the name Bolivia, named after him, you can also include Colombia, America, The Philippines, El Salvador, Dominican Republic and many many more. I have seen several prominent people just shy of outright requesting that.
The Oscars are going to change their eligibility criteria and it will be based supposedly on the British awards, quoting from a NYT article "All entries in two British film categories, outstanding British film and outstanding debut by a British writer, director or producer, are now required to increase representation to meet at least two of four diversity standards, like 'onscreen representation, themes and narratives,' and 'industry access and opportunities.' among others." Do you know what this means for art? Bye Celine, by Kafka,bye Joyce, bye Kurosawa, bye Kubrick, bye Lynch, bye Tarantino and so on. Only wholesome, clean , politically conscious entertainment will be recognized. Now, what does this remind you of?
None of the current power-holders will be affected for all these changes, so no essential change. The big traditional companies are already on-board, Silicon Valley is on board, the savvy political parties are on-board, Hollywood is on-board , the media too. What this will do is to disenfranchise and silence any dissenting, dissonant voice and create a moralist culture which will dictate in a top-down form the "correct way" to do things and interpret the world. You disagree? No Facebook for you, No youtube, no mastercard, no visa, no Stanford, no Netflix, no New York Times, no BBC, nothing for you.
Do you want examples?
A Simon Bolivar statue was vandalized in Florida, now Bolivar was no saint but he is a South-American hero he died in 1830 and now he has been tarnished just by being a "White Slaver"(He was not). Well, let's see you can come now after the name Bolivia, named after him, you can also include Colombia, America, The Philippines, El Salvador, Dominican Republic and many many more. I have seen several prominent people just shy of outright requesting that.
The Oscars are going to change their eligibility criteria and it will be based supposedly on the British awards, quoting from a NYT article "All entries in two British film categories, outstanding British film and outstanding debut by a British writer, director or producer, are now required to increase representation to meet at least two of four diversity standards, like 'onscreen representation, themes and narratives,' and 'industry access and opportunities.' among others." Do you know what this means for art? Bye Celine, by Kafka,bye Joyce, bye Kurosawa, bye Kubrick, bye Lynch, bye Tarantino and so on. Only wholesome, clean , politically conscious entertainment will be recognized. Now, what does this remind you of?
None of the current power-holders will be affected for all these changes, so no essential change. The big traditional companies are already on-board, Silicon Valley is on board, the savvy political parties are on-board, Hollywood is on-board , the media too. What this will do is to disenfranchise and silence any dissenting, dissonant voice and create a moralist culture which will dictate in a top-down form the "correct way" to do things and interpret the world. You disagree? No Facebook for you, No youtube, no mastercard, no visa, no Stanford, no Netflix, no New York Times, no BBC, nothing for you.
It seems like you are drastically overstating what the BFI's actual requirements are[1]. Basically if you hire some people outside of London/South East England and if you offer some paid interships and training to under-represented groups, you have passed.
[1] https://www.bfi.org.uk/sites/bfi.org.uk/files/downloads/bfi-...
[1] https://www.bfi.org.uk/sites/bfi.org.uk/files/downloads/bfi-...
I was just quoting the New York Times article, if what you say it's true then the rule is meaningless and unnecessary and only included to virtue-signal.
[deleted]
This view is an improvement over the status quo view held by most technologist.
The 'technologist' would typically (I would argue) want to remove bias from data, not introduce it to make a model less effective at its intended purpose.
The article suggests making models politically aware, which is probably standard practice in many parts of China.
The article suggests making models politically aware, which is probably standard practice in many parts of China.
> remove bias from data,
I wonder if this is really a possible goal. Might it not be better to be aware of the impossibility of true objectivity, and decide to be strongly opinionated in favor of certain values? Certainly if a machine learning model not only predicts something but acts as an actual agent in the world to affect some goal, it becomes impossible to claim pure objectivity, as the AI itself now is acting to change the world in pursuit of that goal.
I mean, it's hard to argue purely "objectively" that cruelty , totalitarianism and power centralization are "wrong". Any statement about them eventually comes to a moral argument, which means that AI researchers ultimately have to take responsibility themselves for being opinionated moral agents.
"Remove bias" could just be passing the buck for support for entrenching the status quo or worse. Things like democracy, egalitarianism, freedom of thought and opportunity don't just spring up spontaneously, they were hard to fight for and win.
I wonder if this is really a possible goal. Might it not be better to be aware of the impossibility of true objectivity, and decide to be strongly opinionated in favor of certain values? Certainly if a machine learning model not only predicts something but acts as an actual agent in the world to affect some goal, it becomes impossible to claim pure objectivity, as the AI itself now is acting to change the world in pursuit of that goal.
I mean, it's hard to argue purely "objectively" that cruelty , totalitarianism and power centralization are "wrong". Any statement about them eventually comes to a moral argument, which means that AI researchers ultimately have to take responsibility themselves for being opinionated moral agents.
"Remove bias" could just be passing the buck for support for entrenching the status quo or worse. Things like democracy, egalitarianism, freedom of thought and opportunity don't just spring up spontaneously, they were hard to fight for and win.
I believe the article is advocating being aware of the 2nd level effects of our work, such as how the power hierarchy shifts when our software is in use.
> remove bias from data, not introduce it
The problem is in finding the bias, since it's a quantity inferred ex post by checking the discrepancy between reality and expectations. And since we have come to ideologically reject the very idea that there might be reasons for the discrepancy, then it must be all up to bias.
Which I personally think it's stupid and counterproductive, since when you have actual reasons you can act upon them to improve the situation, while "bias" is only useful to play a power game in which anyone who disagrees with you can be called racist and forced to repent and apologize for his/ her "bias".
The problem is in finding the bias, since it's a quantity inferred ex post by checking the discrepancy between reality and expectations. And since we have come to ideologically reject the very idea that there might be reasons for the discrepancy, then it must be all up to bias.
Which I personally think it's stupid and counterproductive, since when you have actual reasons you can act upon them to improve the situation, while "bias" is only useful to play a power game in which anyone who disagrees with you can be called racist and forced to repent and apologize for his/ her "bias".
> The 'technologist' would typically (I would argue) want to remove bias from data, not introduce it to make a model less effective at its intended purpose.
This quote from the OP directly address that attitude: "When the field of AI believes it is neutral, it both fails to notice biased data and builds systems that sanctify the status quo and advance the interests of the powerful. What is needed is a field that exposes and critiques systems that concentrate power, while co-creating new systems with impacted communities: AI by and for the people."
> The article suggests making models politically aware, which is probably standard practice in many parts of China.
While that's a technically correct statement, it misses the point by a mile. Firstly, it's impossible to make non-political models: at a minimum they embed the politics of the status-quo, which is often falsely confused with the absence of "politics" (for the same reason it's hard to see the mountain itself when you're standing on top of it). Secondly, what makes a model designed to embed the politics of the Communist Party of China objectionable is not that it embeds an instance of the class "politics," but rather characteristics of that specific instance.
This quote from the OP directly address that attitude: "When the field of AI believes it is neutral, it both fails to notice biased data and builds systems that sanctify the status quo and advance the interests of the powerful. What is needed is a field that exposes and critiques systems that concentrate power, while co-creating new systems with impacted communities: AI by and for the people."
> The article suggests making models politically aware, which is probably standard practice in many parts of China.
While that's a technically correct statement, it misses the point by a mile. Firstly, it's impossible to make non-political models: at a minimum they embed the politics of the status-quo, which is often falsely confused with the absence of "politics" (for the same reason it's hard to see the mountain itself when you're standing on top of it). Secondly, what makes a model designed to embed the politics of the Communist Party of China objectionable is not that it embeds an instance of the class "politics," but rather characteristics of that specific instance.
> This quote from the OP directly address that attitude
It doesn’t directly address it, the parent was talking about intention, the article is talking about outcome. Smooshing then together produces nonsense: “when someone wants to remove bias from data, they will fail to notice biased data”.
Surely it isn’t a given that someone trying to remove bias will believe that all bias has been removed?
It doesn’t directly address it, the parent was talking about intention, the article is talking about outcome. Smooshing then together produces nonsense: “when someone wants to remove bias from data, they will fail to notice biased data”.
Surely it isn’t a given that someone trying to remove bias will believe that all bias has been removed?
[deleted]
"Fair" and "good" are certainly tricky words to narrow down, granted. But to say they're not even worth discussing - that the only question to ask is whether AI shifts power to groups I like from groups I don't like - is to let politics swallow the entire field and make collaboration across ideological boundaries impossible. That's not going to lead to good outcomes for anyone.
Typically ethics is devoid of politics. A simple apolitical question: is this research going to harm anyone?
It’s open ended and simple enough to answer.
It’s open ended and simple enough to answer.
I'm a bit uncomfortable with this kind of political posturing being published in Nature, the preeminent science journal. It seems the exact wrong forum for it.