Gå frakoblet med Player FM -appen!
S03 E05: OpenAI Disrupts Covert Influence Operations With The Help of OSINT
Manage episode 428102193 series 3491074
Key Points Discussed:
• Monitoring and Disruption Efforts: OpenAI collaborates with open-source intelligence practitioners to monitor internet activity and identify potential misuse of their language models by nation-states and other actors. They aim to disrupt sophisticated threats through continuous improvements in their safety systems and collaboration with industry partners.
• Recent Trends: OpenAI has detected and disrupted operations from actors in Russia, China, Iran, and a commercial company in Israel. These operations, including ones named "Bad Grammar" and "Doppelganger," used AI to generate content but failed to engage authentically with audiences.
• Techniques and Tactics: The actors use AI to produce high volumes of content, mixing AI-generated and traditional formats, and faking engagement by generating replies to their own posts. Despite these efforts, they struggled to reach authentic audiences.
• Defensive Strategies: OpenAI employs defensive design policies, such as friction-imposing features, to thwart malicious use. They also share detailed threat indicators with industry peers to enhance the effectiveness of disruptions.
• Case Studies: Examples include Russian and Chinese networks targeting various regions with limited engagement, and an Iranian network generating anti-US and anti-Israeli content. These operations highlight the ongoing challenge of AI misuse.
• Open Source Intelligence: Dekens discusses his work with Shadow Dragon, including a white paper on using open-source intelligence to identify and monitor troll and bot armies. He explains how prompt error messages can be a key indicator of malicious activity.
22 episoder
S03 E05: OpenAI Disrupts Covert Influence Operations With The Help of OSINT
OSINT with ShadowDragon & Digital Tools For Modern Investigations
Manage episode 428102193 series 3491074
Key Points Discussed:
• Monitoring and Disruption Efforts: OpenAI collaborates with open-source intelligence practitioners to monitor internet activity and identify potential misuse of their language models by nation-states and other actors. They aim to disrupt sophisticated threats through continuous improvements in their safety systems and collaboration with industry partners.
• Recent Trends: OpenAI has detected and disrupted operations from actors in Russia, China, Iran, and a commercial company in Israel. These operations, including ones named "Bad Grammar" and "Doppelganger," used AI to generate content but failed to engage authentically with audiences.
• Techniques and Tactics: The actors use AI to produce high volumes of content, mixing AI-generated and traditional formats, and faking engagement by generating replies to their own posts. Despite these efforts, they struggled to reach authentic audiences.
• Defensive Strategies: OpenAI employs defensive design policies, such as friction-imposing features, to thwart malicious use. They also share detailed threat indicators with industry peers to enhance the effectiveness of disruptions.
• Case Studies: Examples include Russian and Chinese networks targeting various regions with limited engagement, and an Iranian network generating anti-US and anti-Israeli content. These operations highlight the ongoing challenge of AI misuse.
• Open Source Intelligence: Dekens discusses his work with Shadow Dragon, including a white paper on using open-source intelligence to identify and monitor troll and bot armies. He explains how prompt error messages can be a key indicator of malicious activity.
22 episoder
Alle episoder
×Velkommen til Player FM!
Player FM scanner netter for høykvalitets podcaster som du kan nyte nå. Det er den beste podcastappen og fungerer på Android, iPhone og internett. Registrer deg for å synkronisere abonnement på flere enheter.