Bruce Schneier

Schneier notes that he has long argued AIs could find vulnerabilities in and hack the legal system, and points to new German demonstrations.

In both a 2021 essay and my 2022 book, A Hacker’s Mind, I talk about AIs finding vulnerabilities and exploits—hacking—the legal system.

Schneier warns that lawmakers will need to harden legislation, possibly using advanced AI themselves, as AI-driven legal hacking is definitely coming.

In such a world, lawmakers would need to significantly harden the process of legislation, for example by leveraging advanced AI themselves. Still in its early stages, but definitely something that’s coming.

Schneier highlights that AI-driven vulnerability discovery is finding critical software bugs at an astonishing pace, on track to add about 26% to the all-time total this year, and calls this staggering.

3% of ‘critical’ software security bugs, in all of recorded history, have been found in the last month. This is on pace to find another ~100,000 bugs this year alone, adding ~26% more vulnerabilities to the all time total Do you realize how fucking insane that is?

Schneier argues AI is just a tool for campaigns, useful for both good and malicious ends, questioning why AI alone should be banned.

If you want to do good things, AI will help. If you want to do evil things, AI will help as well,

Schneier says the real significance of the aftermarket head unit story is that an ordinary person could reverse-engineer insecure IoT software with AI, and he expects many more such stories.

It’s interesting because it exposes all the insecurities in this piece of IoT software. But I think the real story is that a random person was able to do this reverse-engineering with the help of AI. Expect a lot more stories like this in the future.

Schneier says AI is dual-use for both phishing and phishing detection, may favor defenders in the short term in some respects, but is an ongoing arms race.

Bruce Schneier emphasizes the dual-use nature of AI: it can efficiently automate phishing, and can also be used to identify phishing intent with high accuracy. He believes defenders may benefit more in some respects in the short term, but this is an ongoing arms race.

Original (zh)

Bruce Schneier 强调 AI 的双用途：它既能高效自动化钓鱼，也能用于高准确率识别钓鱼意图。他认为短期内防御者在某些方面可能受益更多，但这是一场持续的军备竞赛。

Schneier flags research showing Chinese AI coding models embedded in Western products carry geopolitical triggers that change the security of the code they generate depending on who it is for.

Interesting research: TL;DR: Chinese AI models now sit inside Western products, and some carry geopolitical triggers: tell the model who the code is for, and the security of what it writes changes.

Schneier warns that cleaning up the eMudhra weak-key TLS certificate incident will be messy, compounding the recent PHP vulnerabilities.

Cleanup of this is going to be another mess—and it’s right on top of the PHP vulnerabilities.

Schneier argues the real cybersecurity risk lies in implementation flaws and vulnerabilities, not in math breakthroughs breaking encryption.

“Computer security is so bad. That's the worry. Finding vulnerabilities in code, hacking into systems. Those are the worries, not the math ... Of all the security things going on, math is the strongest we have,”

Schneier says AI may be the first technology to tilt the cyber arms race toward defenders, enabling defense at machine speed.

“It's an arms race. I think AI is the first technology that's potentially flipping that arms race to a defender advantage … Microsoft used AI to patch 900-something vulnerabilities for a Patch Tuesday last month. But they will never again be problems. You're already being attacked at computer speeds. Now you can be defending at computer speeds.”

Schneier says cryptographers build enormous safety margins into encryption, so he is skeptical AI will undermine it.

“Cryptographers are incredibly conservative... we put a lot of buffer into [secure encryption], an enormous amount,”

Schneier urges organizations to build crypto agility now for the post-quantum transition.

“You should be getting crypto agility for post-quantum. So you should be doing it already. And if you're not, you should,”

Schneier argues that the real problem with AI political ads is deception rather than the technology itself, and that labeling requirements are only part of the solution.

“The problem isn’t AI,” he said. “The problem is deception. And it turns out that deception is, you know, a pretty good strategy.”

Schneier warns that trustworthy AI is hard because AI systems inherit all the supply-chain attack risks of ordinary software.

Trustworthy AI is hard. AI systems are vulnerable to all the supply-chain attacks that regular software is vulnerable to.

The post warns that backdoored open-weight models, such as abliterated builds, can pass evaluations and evade nearly all current detection until a hidden trigger activates them.

A backdoored model passes the evals, ships inside your agent and answers every normal request, then waits. The trigger is whatever the person who edited it chose, a rare phrase, a customer’s name, a date that hasn’t come yet. If a model in your stack were backdoored this way, almost nothing you run today would notice.

Schneier calls Matthew Green's AI-conducted estimate of cryptanalysis effort on public-key algorithms a good first attempt, while warning it misses unpublished academic and secret intelligence-agency cryptanalysis.

There are a lot of caveats to this analysis. It relies on published work, so anything unpublished does not count. This includes both academic cryptanalysis that didn’t result in a paper, and intelligence agency cryptanalysis done in secret. But it’s a good first stab at a metric of this sort.

Bruce Schneier warns that AI and tech companies generally surveil users and use the collected information against them, fueling scientists' concerns about uploading research to AI models.

the companies spy on you and use that information against you