Hi everyone,
First off, I owe you all an apology. Last week was rough on me health-wise, and despite my best efforts, I couldn't send out the newsletter. I even drafted it, but couldn't cross the finish line. Sorry for the ghosting, and thanks for sticking around. š
This week, we're bouncing back with a heavyweight issue, packed with stories that underline the ever-evolving dance between cybersecurity and AI.
Let's dive in!
Table of Contents:
The Deepfake Deception Dilemma š
AIās Achilles Heel - Prompt Injection š”ļø
AI-Generated Voices in Robocalls Now Illegal š«
The Threat of AI in Warfare š
OnlyFakeās Frighteningly Real IDs š
ChatGPT Account Takeover Exposed š„
Freelancer Faux Pas: AI in Job Scams? š¤
Innovative Defense: AudioSealās Watermarking Wonders š
Tackling AI Safety: Gradient-Based Language Model Red Teaming šÆ
šÆ Highlights of This Edition:
1. The Deepfake Deception Dilemma š
A multinational firm in Hong Kong was scammed out of HK$200 million (~ $25.5 Million) thanks to deepfake tech.
Imagine sitting on a video call, and the person you're talking to looks and sounds exactly like your CFO, but it's not really your CFO.
This isn't sci-fi; it happened. It's a stark reminder of how real the threat of deepfakes has become.
The sophistication of this scam is a wake-up call for all of us in cybersecurity; We donāt have any real defenses (I mean technical defenses) against these attacks yet; however it doesnāt mean we are totally defenseless either:
Add more checks (in-person checks, multiple signatures/approvals, secondary secure channel confirmations, etc) to the decision-making critical paths in your organization.
Does it slow things down? Yes.
Is it inconvenient? Also Yes.
But do you know whatās even more inconvenient? Losing 25 Million Dollars.
Who wouldāve thought one day āmore bureaucracyā would be the (hopefully temporary) answer? sigh
Another thing Iām wondering about is the infrastructure for an attack like this. What does it look like? The preparation, software, and hardware stack, and the execution, how does it work exactly?
Drop me a line if you know!
Read more about this incident.
2. AIās Achilles Heel - Prompt Injection š”ļø
Prompt Injection is a big deal in LLM security. It's tricking AI into saying or revealing things it shouldnāt.
These two articles dive deep into how we can protect against it, emphasizing the need for roles-based APIs and secure system prompt designs. The author also included curl commands you can use with OpenAI API to test things yourself!
It's a must-read for developers and cybersecurity pros - donāt miss it.
Improve LLM Security Against Prompt Injection - Part 1, Part 2.
3. AI-Generated Voices in Robocalls Now Illegal š«
The FCC's move against AI-generated voices in robocalls marks a significant step towards clamping down on misleading communications.
Iām so happy to see governments and legislative bodies are not only moving fast to respond to AI Security needs, but theyāre also doing it effectively. Cheers to that, and more to come!
4. The Threat of AI in Warfare š
This paper explores how AI could escalate conflicts in military and diplomatic arenas. It seems like AI agents might lean towards escalatory actions, raising red flags about their integration into high-stakes decision-making.
One interesting part (that made me laugh, and also scared me) was the agentsā āsudden, hard-to-predict spikes of escalationā with launching nuclear attacks being a very feasible option.
I guess we need to retrain them on anger management data. or maybe add a ātherapistā layer. sheesh.
5. OnlyFakeās Frighteningly Real IDs š
OnlyFakeās neural network is churning out fake IDs so real they could pass for your driver's license. At $15 a pop, itās an affordable service to bypass KYC.
Not a bit surprised about this - this will only get worse.
6. ChatGPT Account Takeover Exposed š„
Well, this is not exactly āAI Securityā, it falls more under web security.
Iām including it for two reasons, 1- it affected an AI-related product, and 2- as a reminder that vulnerabilities from all other attack surfaces are still very much relevant.
7. Freelancer Faux Pas: AI in Job Scams? š¤
A developerās tweet revealed a sneaky trend: freelancers using AI to auto-respond to job postings, without reading them. (Again, anyoneās surprised?)
James Potter, owner of meals.chat (an app that tracks your macro and calories by looking at the photos of your food), posted a job on a freelance platform.
He, however, added a small detail to the job description:
And he caught one!
Combine this with that invisible prompt injection technique I covered in previous issues and watch the stream of AI-generated responses tell on themselves!
8. Innovative Defense: AudioSealās Watermarking Wonders š
Meta's AudioSeal uses watermarks to tell if an audio clip is AI-generated, adding a layer of security against voice cloning.
See? Weāre getting there, weāre making progress. One step at a time.
It may not be very practical right now since scammers are not going to watermark their audio and deepfakes, but itās a great step in the right direction.
Discover AudioSealās innovation
You can also see the code, and even install it here.
9. Tackling AI Safety: Gradient-Based Language Model Red Teaming šÆ
Google and Anthropic are pushing the envelope with Gradient-Based Red Teaming (GBRT), an automated approach to discover vulnerabilities in AI models. GBRT leverages the model's own gradient information to generate prompts that hit the right spot. This is huge since it has the potential to partially, if not completely automate the process which leads to better, faster, and more comprehensive testing.
Check out the research
GitHub Repo
š Call to Action:
Don't let the conversation end here!
If you found value in this issue, consider sharing ContextOverflow with someone whoād appreciate it - itād mean a lot to me.
Stay tuned for more insights, and stay secure!
Until next time,
Samy