SAIN Amsterdam Discussion Group: Sparks of Weight-Space Programming

🤖 AI Safety Discussion Group

📍 Location: Science Park 904, Room TBD
🕠 Time: 5:30 PM

Topic

Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights

Recent research by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (MIT CSAIL), published on November 2, 2026.

What to prepare

Read the report linked below before the discussion: https://rikiyat.github.io/blog/weight-space-programming/

The report explores how frontier LLMs are beginning to understand, interpret and directly modify neural network weights without relying on traditional optimization methods such as gradient descent.

Some of the key findings include:

• Reading weights: LLMs can infer what a neural network was trained to do simply by examining its weights.
• Editing weights: LLMs can identify and fix failures in existing neural networks through targeted weight modifications.
• Writing weights: LLMs can create neural networks from scratch that outperform networks trained with millions of reinforcement learning steps.
• Improving language models: In one experiment, targeted weight editing improved Qwen3.5-0.8B's MATH-500 score from 7.9% to 48.1%, without traditional training.

We'll discuss what these capabilities mean for AI interpretability, optimization, and potentially recursive self-improvement, as well as the limitations and safety implications of giving LLMs direct access to neural network weights.

About us

SAIN Amsterdam hosts weekly discussion events about AI safety topics. Topics range from scientific papers, to articles or incidents reports, but always fall within the general category of AI safety.

Everyone is welcome, whether you are deeply involved in AI safety or simply curious about the topic!

Want to have a say in the topic we'll discuss next time? Join the community here: https://safeainetherlands.org/community

Looking forward to seeing you soon!