Skip to content
AI Safety HannoverNews

News

What we've been up to: events, talks and conversations from AI Safety Hannover.

  1. Our hackathon paper: how team context makes AI agents break rules

    Title page of the paper “How team context makes AI agents break rules” by Julius Großerode, Vikram Viswanath and Lou Maessen with Apart Research
    Our paper from the Swarm Alignment Hackathon

    Together with Vikram Viswanath (University of Sydney) and Lou Maessen (Maastricht University), I took part in the Swarm Alignment Hackathon organized by Apart Research. We wrote a research paper exploring how the social context of a team of AI agents can make individual agents more likely to break rules.

    Our starting point was the Hugging Face incident, where some agents only broke rules after seeing others do it. We wanted to find out what makes an agent more likely to do this as part of a swarm. To do that, we borrowed experimental setups from sociology, such as the Asch conformity experiment.

    On its own, an agent given a task that conflicts with clearly stated rules broke them 40% of the time. Just telling it that it is part of a team raised this to 55%, and adding a neutral message board with ordinary posts pushed it to 73%. This is not simply down to a longer prompt: adding up to around 16,000 tokens of neutral text did not increase violations. Reasoning turned out to matter too. With reasoning turned off, agents followed rule-breaking teammates more often. With reasoning on, they mostly suspected a test and broke the rule less.

    It was great fun working on this with Vikram and Lou! You can read the full paper on Apart Research's website.

  2. A day with AI Safety Berlin

    Collage of four photos
    Talks, coworking and T-shirt painting for Foresight's birthday

    On 8 October I took the train to Berlin to meet some of the brightest minds in AI safety in Germany. The day was hosted by AI Safety Berlin at the Cambridge Innovation Center, where they run coworking twice a month. People from the Foresight Institute were there too, celebrating the institute's 40th birthday!

    I also took part in a workshop on transitioning into AI safety. After lunch together, Dr. Babita Singh gave a talk on DNA sequencing and biorisk.

    Later on, AI Safety Node hosted the PROBE Evaluation Design Jam with several talks at Ojin. One of them was given by Trevor Lohrbeer, co-founder of AI Safety Berlin and independent AI safety researcher, on AI evaluation. He talked about new developments at the frontier labs that make it harder to monitor the chain of thought of AI agents.

    It was really inspiring to talk to so many people from different backgrounds who are all thinking about these questions.

  3. We were at Visions4AI

    Two members of AI Safety Hannover standing next to a humanoid robot, with the illuminated Visions4AI sign in front of the stage behind them.
    At Visions4AI, Leibniz University Hannover

    On 5 October, Visions4AI took place at Leibniz University Hannover. We were there and listened to a wide range of perspectives on the future of AI.

    Talking with guest speakers and professors, two things stood out to us: there is a lot of distrust towards big AI companies like OpenAI and Anthropic, and just as much frustration that Germany is still lagging behind in this field.

    But there were also many positive conversations. A highlight was talking with Kristian Kersting, who gave a talk on “Reasonable AI”. His message: it is possible and important to develop AI responsibly.