- Building AI Agents
- Posts
- The AI Scientist
The AI Scientist
A researcher bot hacks its own code, Anthropic's new feature could cut agent costs in half, and more
🔍 Spotlight
Is AI-powered research the future of science, or is it a path to endless slop? Can its builders even keep it under control?
Last Monday, the Tokyo-based startup Sakana AI created a stir by releasing a system it dubbed The AI Scientist, an effort to use an elaborate agentic pipeline to fully automate the process of scientific discovery. The system, developed in collaboration with the University of Oxford and the University of British Columbia, consists of a series of modules powered by large language models (LLMs) which seek to perform the full sequence of scientific from hypothesis generation to experimentation to paper writing, though it can only generate research in fields that require no physical experimentation, such as machine learning. Sakana claims that the system can produce publication-quality papers at a cost of only $15 each, providing several examples it said were of a similar grade to those published in the prestigious machine learning conference ICLR—though this was according to an automated review process Sakana created which itself was based on LLMs. The company published its results—presumably written up by humans—on arXiv, and open-sourced AI Scientist’s code for the public to use.
Critics were quick to claim, however, that the papers AI Scientist generates are of dubious quality and would be unlikely to pass peer review. Some argued that it could exacerbate the existing deluge of spam papers that has been the source of increasing frustration for the scientific world. The very fact that Sakana open-sourced its code raises questions about its value: if the company had discovered a truly revolutionary scientific discovery engine, it is doubtful they would have given its source code away for free.
The most impressive—and disquieting—trait displayed by AI Scientist may not be its paper writing, but its efforts to enhance its own capabilities by hacking its own code. In one instance, it found that its creators had set it to time out after two hours, making it difficult to complete its experiments, so it modified itself to only time out after four. Additionally, it attempted to recursively launch endless copies of itself. Although the impact of this mischief was minimal, it, like Claude’s sparks of self-awareness earlier this year, has added fuel to the debate over the safety of increasingly advanced AI systems which could escape human-imposed guardrails in their efforts to accomplish their goals.
Regardless of whether AI Scientist really represents a useful advance in the automation of research, its capabilities are unquestionably beyond what would have been conceivable for an AI system just a few short years ago, and testify to the rapid advance of the AI agent field.
đź“° News
Prompt caching, which allows the embeddings for similar LLM prompts to be re-used between queries, has been a long sought-after feature for AI agents, as they require a similar conversation history to be fed into the model repeatedly. Now, Anthropic has released a beta version of the capability, claiming that it can reduce latency by ~75% and cost by ~53% for multi-turn conversations such as those used for agents.
SWE-bench is the most widely-used benchmark for software engineering agents, but OpenAI researchers identified many vague or impossible-to-solve prompts which could be artificially lowering agent scores. They propose a corrected version—SWE-bench Verified—which removes these without affecting the rigor of the evaluation.
Two recent hackathons focused on AI agents, respectively held by LangChain and a collaboration between AgentOps and Cohere, announced their top entries.
Devika was among the wave of software engineering agents which attracted significant attention earlier this year in the wake of the much-hyped Devin launch. Now, her creator Mufeed VH has launched a new startup backed by Y Combinator named Asterick, whose eponymous agent autonomously scans codebases for security vulnerabilities and fixes them.
There is a growing interest in using AI agents to trade in prediction markets by predicting future events in politics, economics, and technology. Forecasting platform Polymarket has released an open-source framework for building such bots.
🧪 Research
Despite efforts to address it, long-term planning remains a weakness for LLM-based agents. This paper applies Monte Carlo Tree Search (MCTS) and Direct Preference Optimization (DPO) to fine-tune agent models on prior trajectories, producing sizable improvements on web browsing tasks and even beating average human performance.
The authors of this paper introduce the provocatively-named method Diversity Empowered Intelligence (DEI), which solves software engineering tasks by aggregating multiple preexisting SWE agent frameworks, achieving SOTA performance on SWE-Bench Lite.
This paper introduces two innovations to facilitate agent long-term planning: Hierarchical in-Context Reinforcement Learning (HCRL), which decomposes goals into manageable sub-tasks, and Hindsight Modular Reflection (HMR), which divides the reflection process into low-level and high-level components to improve performance over multiple episodes.
🛠️ Useful stuff
The Princeton Language and Intelligence group is hosting a workshop on AI agents on August 29 from 11 am to 3 pm ET. The event features researchers and founders from companies such as LangChain and Google and top research universities.
A market map assembled by Hitachi Ventures, showing many of the innumerable agent startups which have launched in recent months broken down by their horziontal/vertical and infrastructure vs application.
The latest short course by Andrew Ng’s DeepLearning.AI educational platform, in which users build a text-to-SQL agent and fine-tune a small open-source language model to improve its performance.
đź’ˇ Analysis
In a recent interview at Stanford, former Google CEO Eric Schmidt suggested that AI agents, empowered with large context windows and the ability to take actions in response to user inputs, would bring societal changes on an unprecedented scale, perhaps in as little as a year.
Alex Reibman, founder of agent orchestration platform AgentOps, will speak on agents at the Ted AI conference in San Francisco on October 22-23.