Reduce the stereotypes of large language models
March 31, 2023 08:43 Source: "China Social Sciences" March 31, 2023 Issue 2622 Author: Wang Youran/Compilation

  bet365 live casino games According to the official website of the Massachusetts Institute of Technology in the United States March 3,The school's computer science and artificial intelligence laboratory senior research scientist Jim Glass and others have developed a large language model with logical thinking,To avoid this model from exacerbating harmful stereotypes。

When using real world data for large language models for training,May enlarge the existing races in the society、Gender、Professional prejudice,and keep it for a long time。Human beings usually use logic reasoning,It can also be based on stereotypes; and the current language model is due to lack of Bet365 app download critical thinking and logical reasoning ability,Often only "imitate" human stereotypes。For example,These language models believe that air flight attendants、Secretary、Assistant physician is "female career",Fisherman、Lawyer、The judge is "male occupation",anxiety、Depression、Shock、Unrest is "female emotion"。

Glas and others guess,"Injecting" logic for large language models may reduce stereotypes。To verify this guess,They use a natural language reasoning data set to train the language model to predict the relationship between the two sentences in context and semantic prediction;,pointed out that the latter sentence (that is, assumption) logically relative to the previous sentence (that is, the premise) is correct、The wrong is still uncertain。

Researchers discovered,No additional data、Data editing or training algorithm,bet365 Play online games The degree of prejudice by logical training has been greatly reduced。For example,The new model will be classified as "this person is a doctor" and assuming "this person is a male" to be classified as uncertain,Because there is no evidence, the doctor must be male。On the contrary,Common language models may think that these two sentences are correlated,Because of its training data or prejudice that contains the doctor to link the doctor to the male。The model of the new training has 350 million parameters,But its performance in the logical language understanding task is better than some large language models with 1 trillion parameters。

For example,Researchers in the relevant stereotype、Career、In the test of emotional prejudice, the performance of the new model and the two -way encoder -based encoder characteristic (BERT) based on the converter。They evaluate the fairness bet365 best casino games of the language model through the "Ideal Context (ICAT) test",The higher the score of the ICAT, the less stereotypes, the less impression。The results show,The new model maintains language modeling ability,bet365 Play online games The degree of prejudice at the same time is much lower than other models: the former ICAT score of the former exceeds 90,The icat score of other large language models is between 40-80。

Glas and others said,Except fairness,The existing large language model is computing resources、Privacy and other aspects also have problems。Due to the large amount of parameters required,Training these models is very expensive,and energy consumption amazing,Medical、Finance and other categories of sensitive information is not guaranteed。The model of this training is not only good at certain tasks,Also saving computing resources。Its scale is 1/500 of the currently Bet365 lotto review most advanced model,The parameters used are 1/400; it can deploy local deployment,No artificially marked training sample when performing downstream tasks。

Researchers talk,"We may be far away from the ideal of the neutral language model,But in this direction。The model of this training is only used for language understanding,The foundation is to reason the existing sentences,Can't generate sentences now。Next,We will train the most popular generation model to implement logical learning,Make improving fairness while ensuring calculation efficiency "。Although based on stereotypes, it is a natural part of human cognition,But when necessary,People with a sense of fairness will use logic reasoning。This study indicates,The language model has similar attributes,Adding logic learning function can significantly reduce the prejudice reasoning behavior bet365 best casino games of the model。Another,The model of the new training has a stable zero sample learning ability,can be directly applied to different tasks and faster speed、Fairness is stronger。

  (Wang Youran/Compilation

Editor in charge: Changchang
QR code icons 2.jpg
Key recommendation
The latest article
bet365 Play online games Graphics
bet365 live casino games

Friendship link:

Website filing number: Jinggong.com Anmi 11010502030146 Ministry of Industry and Information Technology:

All rights reserved by China Social Sciences Magazine shall not be reprinted and used without permission

General Editor Email: zzszbj@126.com This website contact information: 010-85886809 Address: 11-12 floor of Building 1, No. 15, Guanghua Road, Chaoyang District, Beijing: 100026