Flux
Toutes les catégories

IA

1664 articles

The first known runaway AI agent - or a very bad marketing stunt?
IA Programmation

The first known runaway AI agent - or a very bad marketing stunt?

The first known runaway AI agent - or a very bad marketing stunt? Martin Alderson's commentary on the OpenAI accidental cyberattack against Hugging Face includes a couple of details I hadn't considered. First, Hugging Face offers a truly rich target if you're trying to find potential vulnerabilities that require executing arbitrary code: Hugging Face has an enormous attack surface. They have more interfaces than I can count which run untrusted models and code. While they definitely have…

Simon Willison's Weblog
You Probably Won’t Read This Article…and That’s OK
IA

You Probably Won’t Read This Article…and That’s OK

“Help! There are too many [LLM bug reports, blog posts about LLM bug reports, books, treatises, codices, scrolls, papyri, cuneiform tablets]! How do I choose which to read?” —Many people, presumably Stop there! If you are reading this, ask yourself how you got here. Did Substack’s algorithm recommend this article for you? Did a juicy […]

O'Reilly Radar — AI/ML
Quoting Seth Larson
IA Programmation

Quoting Seth Larson

The Python Package Index (PyPI) now rejects new files being uploaded to releases that are older than 14 days. This restriction was put in place to prevent old and long-stable releases from being poisoned in case publishing tokens or workflows of PyPI projects were compromised. As far as we are aware this has not yet been abused, but there is no technical reason beyond that attackers weren't aware it was possible. — Seth Larson, PyPI blog Tags: packaging, python, supply-chain, pypi,…

Simon Willison's Weblog
Quoting Thomas Ptacek
IA Programmation

Quoting Thomas Ptacek

I genuinely believe that if you took an open weights model from 2025 and built a pentest harness for it, it could do this kind of sandbox escape and scan/hack in most networks. This is only surprising because you assume OpenAI has sounder sandboxes. — Thomas Ptacek, doesn't think this even needs a frontier model Tags: thomas-ptacek, openai, security, generative-ai, ai-security-research, ai, llms, sandboxing

Simon Willison's Weblog
OpenAI’s accidental cyberattack against Hugging Face is science fiction that happened
IA Programmation

OpenAI’s accidental cyberattack against Hugging Face is science fiction that happened

This story is wild. The short version: OpenAI were running a cybersecurity test against an unreleased model, with the model's guardrail features turned off. Rather than solve the test, the model broke its way out of OpenAI's sandbox, then found exploits to break in to Hugging Face, all so it could cheat on the test by stealing the answers. Along the way it helped make the strongest case yet for how the imbalance of model availability is hurting our ability to secure our software. Here's what…

Simon Willison's Weblog
Are AI labs pelicanmaxxing?
IA Programmation

Are AI labs pelicanmaxxing?

Are AI labs pelicanmaxxing? Excellent piece of work by Dylan Castillo, who took a deep-dive into the frequently pondered question of whether the AI labs have been deliberately training models to draw pelicans riding bicycles in response to my deeply unscientific benchmark. I've been randomly spot-checking this in the past by testing models against other animals riding other types of vehicle, but never with anything close to the diligence of Dylan's methodology here. Dylan took 8 animals × 6…

Simon Willison's Weblog
Stop Overengineering Your Agent Harness
IA

Stop Overengineering Your Agent Harness

The following originally appeared on Hugo Bowne-Anderson’s Vanishing Gradients Substack and is being republished here with the author’s permission. The conversation around harness engineering is dominated by problems from coding and personal agents such as OpenClaw, but most agents are simpler. Builders should avoid over-engineering for capabilities that newer models may absorb anyway, the “Kirby […]

O'Reilly Radar — AI/ML
Orchestrions
IA Programmation

Orchestrions

San Francisco tip: it only costs around $15 ($10 in quarters plus a $5 bill for the self-playing violin) to activate every single Orchestrion in Musée Mécanique. And because most people are bad at allocating their funds you may well be the ONLY person activating the Orchestrions, which means you get to craft the soundscape for the entire museum. Tags: san-francisco

Simon Willison's Weblog
Nativ: Run AI models locally on your Mac
IA Programmation

Nativ: Run AI models locally on your Mac

Nativ: Run AI models locally on your Mac Prince Canuma is the developer behind the excellent MLX-VLM Python library for running vision-LLMs using MLX on a Mac. I'm really excited about his new project, which wraps MLX in a full macOS desktop application. It's similar in shape to LM Studio, providing both a chat interface and a localhost API server for accessing models. The app picked up MLX models I had already tried that were present in my Hugging Face cache directory, which was a nice touch.…

Simon Willison's Weblog
A Fireside Chat with Cat and Thariq from the Claude Code team
IA Programmation

A Fireside Chat with Cat and Thariq from the Claude Code team

Earlier this month I hosted a fireside chat session at the AI Engineer World's Fair with Cat Wu and Thariq Shihipar from Anthropic's Claude Code team. We talked about Claude Code, Claude Tag, Fable, coding agent security, evals, tool design, and how Anthropic use these tools themselves. The full video of the session is now available on YouTube. Below is an edited copy of the transcript, with extra links and my own bolded highlights. A few top-level notes if you don't want to watch the video or…

Simon Willison's Weblog