OpenAI, the renowned artificial intelligence (AI) research organization, has unveiled its latest initiative – the OpenAI Red Teaming Network. This contracted group of experts will play a crucial role in enhancing the robustness of OpenAI’s AI systems by assisting in the assessment and mitigation of model risks.
Key Takeaway
OpenAI has launched the Red Teaming Network, a contracted group of experts, to enhance the robustness of its AI models. Red teaming plays a crucial role in identifying biases and evaluating safety filters in AI systems. While some experts propose alternative approaches like violet teaming, red teaming networks remain a valuable step towards addressing model risks.
Why Red Teaming Matters
Red teaming has emerged as a pivotal step in the development process of AI models, particularly as generative technologies gain wider adoption. By subjecting models such as DALL-E 2, which has faced criticisms for perpetuating stereotypes, to red teaming, biases can be identified and addressed. Additionally, text-generating models like ChatGPT and GPT-4 can be evaluated for their adherence to safety filters.
Although OpenAI has previously collaborated with external experts through its bug bounty program and researcher access program, the newly launched Red Teaming Network formalizes these efforts. OpenAI aims to strengthen and broaden its collaboration with scientists, research institutions, and civil society organizations to ensure the reliability and safety of its AI models.
Expanding the Network
OpenAI emphasizes that its Red Teaming Network complements external governance practices, such as third-party audits. Experts within the network will be engaged based on their specific expertise to participate in red teaming activities at various stages of model and product development.
Furthermore, members of the Red Teaming Network will have the opportunity to collaborate with each other on general red teaming practices and share findings. OpenAI acknowledges that not all members will be involved in every project, and time commitments will be individually determined, ranging from as few as 5 to 10 hours per year.
The organization actively encourages a diverse range of domain experts to participate, including those from linguistics, biometrics, finance, and healthcare. Prior experience with AI systems or language models is not a mandatory requirement for eligibility. However, participants may be subject to non-disclosure and confidentiality agreements that could impact other research endeavors.
Can Red Teaming Alone Suffice?
Some experts argue that red teaming alone may not be enough. Aviv Ovadya, a contributor to Wired and affiliated with Harvard’s Berkman Klein Center and the Centre for the Governance of AI, advocates for “violet teaming.” This approach involves leveraging the same system, such as GPT-4, to identify potential risks and develop tools to defend institutions and public welfare. While this idea has merit, the lack of incentives and the need to slow down AI releases pose challenges to its implementation.
For now, red teaming networks like OpenAI’s appear to be the most viable solution in ensuring the robustness of AI systems.
