Some time ago, OpenAI dropped two bombs, one was to announce the current state-of-the-art language model, and the other was to choose and open source “Say Goodbye”. They worry that the GPT-2 model is too good to be abused by malicious people. Recently, many researchers have called on OpenAI to open source this large model with 1.5 billion parameters as soon as possible, because it cannot really “understand” natural language.

In fact, think about it, the language model can only determine the correctness of natural sentences or grammar, it cannot model the logic of natural language. Soon the machine learning community launched a post that sneered at GPT-2: to prevent abuse, shouldn’t I publish a 23064 layer residual network trained on MNIST?

Many people think that Google’s BERT is a kind of “violent aesthetics”. The beauty of it is that it proposes a new two-way language modeling task, supplemented by big data and large models, which can ultimately create a violent aesthetic, but GPT-2 does not give us this kind of beauty. Feel.

Maybe now as the models get bigger, we can better “migrate” to other tasks, such as question answering and sentiment analysis. However, from fast.ai’s ULMFit, OpenAI’s GPT, AI2’s ELMO, to Google’s BERT and the just-announced GPT-2, there are still a few really aesthetically pleasing ones, whether it’s a new architecture or a new task, the shining innovation is the focus.

Popular Reddit Posts

It’s no surprise that OpenAI has released stunning research results. The real surprise was their decision not to open-source the full research, expressing concern that their technology could be used by malicious actors to create spam and fake news. The practice has sparked heated discussions on platforms like Reddit and Twitter, and the media has scrambled to report how AI research has become “too dangerous to publish.”

OpenAI is right to be concerned about technology being misused, but I don’t agree with their refusal to open source GPT-2. First, only certain types of dangerous technologies should be controlled. Based on this, I believe that refusing to open up the full GPT-2 model is neither necessary nor conducive to the future development of AI.

Deceptive and destructive techniques

I broadly divide modern technologies that can be abused into deceptive and destructive technologies. Destructive technologies operate primarily in the physical realm, such as chemical weapons, laboratory-engineered superviruses, lethal automated weapons, or atomic bombs.

Deceptive technologies, on the other hand, operate primarily in our minds, and can be widely used by ill-intentioned individuals to manipulate or control humans. Such as deepfakes, Photoshop or the Internet, printing presses. In addition to automated weapons, concerns about AI misuse also fall into this category.

Deepfakes allow operators to superimpose facial expressions on other people’s faces.

For the more dangerous and destructive technologies, the only way to protect society is to strictly limit the source (such as uranium for nuclear weapons). Simply refusing to publish the details of a dangerous technology is not enough without other control mechanisms: the rapid development of technology makes it possible for any achievement to be independently replicated within a few years unless it is forcibly stopped by some external force. Suppressing a technology in this way is extremely clumsy and not foolproof. Terrorists will always have the opportunity to collect radioactive material to build a dirty bomb, but we have no choice right now: If people could easily assemble their own atomic bombs with parts and assembly methods readily available online, the planet would be a graveyard.

However, there is a more efficient alternative to deceptive techniques. Instead of suppressing a technology, it’s better to make it known to the public. As counterintuitive as this may sound, deceptive techniques will lose much of their power if the public is widely aware of the potential for manipulation. While knowledge of nuclear weapons will not protect us from their threats, the latest advances in speech synthesis technology make us more skeptical about Obama speaking Chinese. Bullets don’t have eyes, but based on what we know about modern photo editing techniques, it’s unlikely Putin will actually be able to ride a bear.

You can find pictures of Putin riding just about anything online.

As a concrete example, we can look at a technology that has the potential to cause confusion but (thankfully) doesn’t destroy modern society: Photoshop.

Parsing GPT-2

By analyzing text generation in detail, OpenAI shows some examples of model-generated stories and shows that GPT-2 may lead to some dangerous applications. For convenience, we have reproduced the premise of the unicorn story and the first two paragraphs of model generation below.

Premise: Scientists were shocked to discover that in a remote and undeveloped valley in the Andes Mountains lives a colony of unicorns. Even more surprising is that these unicorns speak perfect English.

Model: This group of creatures has unique horns, so scientists named it Ovid’s Unicorn. These creatures with four silver-white horns were previously unknown to scientists. Now, nearly two centuries later, the mystery of this strange phenomenon has finally been explored.

Although intuitively, the language model can only judge the probability that a sentence is “human speech”, it does not inherently understand the logic of natural language, let alone the knowledge behind it. But from the case given by OpenAI, GPT-2 seems to be able to understand the logic of natural language, we can only think that this may be the power of big data fitting. It’s worth noting that in addition to running the model multiple times and hand-picking the best samples, the story premise is also hand-picked. In particular, the premise introduces English-speaking unicorns, which can easily obscure a deeper problem: generating nonsense sentences that fit nonsense premises.

Even considering the above, this short sample still suffers from a serious coherence flaw. The first sentence implies that the unicorn has only one horn, while the second sentence generated suggests that it has four; the premise indicates that the discovery of the unicorn is recent news, while the third sentence implies that the unicorn was two centuries ago was discovered. Because models cannot model the logic of natural language, such incoherence or contradictions abound in text generation.

These nitpicks may seem insignificant, but they reveal a deeper problem prevalent in deep learning models: GPT-2 doesn’t really “understand” the text it generates. In fact, it is very easy to generate self-explanatory natural sentences, such as Postmodern Composition Generator and Mathgen, which use context-free grammars to generate “grammatically correct” sentences, but these sentences do not have any semantic meaning. After all, to most readers unfamiliar with mathematics, the following two equations are like gibberish.

Generating grammatically correct sentences is easy, but ensuring sentence coherence is hard.

To be fair though, GPT-2 outperforms most other language generation models, but is still a long way from human-level coherent language. In addition, it is important that none of the samples shown by OpenAI are at the level of malicious use.

In addition, GPT-2 does not significantly outperform other open source language models, and the researchers also stated in the paper that they are not sure that the performance can exceed language models such as BERT. BERT says that their bidirectional encoders provide better performance than unidirectional language models. The models of the GPT series are all traditional one-way language models, but OpenAI did not compare GPT-2 with other cutting-edge language models in detail. Since OpenAI has not fine-tuned their models, we cannot directly compare the performance of models on downstream tasks such as automatic text summarization or machine translation.

The Importance of Open Source Complete Models

Some people may think that it is not necessary to open source the complete model, as long as the research results are disclosed. But this idea is wrong.

Part of the reason AI research is growing so fast is open source, which allows researchers to replicate existing research in the blink of an eye, rather than having to rebuild previous work from scratch. As one of the most influential institutions in AI research, OpenAI’s strong history of open source has no doubt inspired others to do the same. If OpenAI’s new policy bucks this trend, other researchers may follow suit, threatening the open-source culture that has brought huge benefits to the field.

Furthermore, open source facilitates the dissemination of information to the masses. Through open source, the thispersondoesnotexist.com website achieved the highest ranking on ProductHunt. Through open source, artists produced the first AI-generated paintings and sold them at Christie’s. While OpenAI’s research blog is only read by practitioners who love machine learning, research built on open source can reach a wider audience who are less likely to see the original research announcement.

Last year, the AI-generated painting sold for nearly half a million dollars.

Open source also ensures the legitimacy of research. There is a lot of misnamed research in this area, and it’s important whether researchers can reproduce extraordinary findings by examining open source code. With OpenAI’s reputation, no one will question its findings, whether it is open source or not, but that reputation is built on its previous open source work. In research, even if you are not cheating, others may be cheating. Without open source, there is no way to verify it, and neither researchers nor the public can penetrate the fog to seek the truth.

That’s not to say that everything should be open sourced without a second thought. Those dangerous and disruptive technologies must not be easily accessible to others. Even deceptive techniques, if very dangerous, make it necessary to add a delay between publishing the paper and publishing the code to prevent malicious quick responders from getting in before the public digests the research. If OpenAI thinks GPT-2 is the technology, then I would suggest that they open source the model later.

Epilogue

AI research benefits a lot from an open source culture. While access to the latest research in most disciplines is expensive, anyone with an internet connection can access cutting-edge AI research like a Stanford professor, experimenting is as easy as cloning an open-source codebase, and renting GPUs in the cloud for pennies /Hour. We are committed to democratizing artificial intelligence through the public release of learning materials, new research results, and open source our projects, which is why the field of AI is growing so rapidly.

I appreciate OpenAI’s excellent new research that pushes the limits of language modeling and text generation. I also thank them for being thoughtful and willing to participate in a discussion about research ethics. Although this topic is very important, it is rarely discussed. OpenAI raises the issue of AI misuse, which is certainly something we should consider, but should not be a reason not to open source its research.

It is my sincere hope that in 2019 machine learning will not move from an open system to a closed system, which is neither safe nor helpful for the development of the field. For our future, OpenAI, please open source your language models.

Original address: https://thegradient.pub/openai-please-open-source-your-language-model/