Open Philanthropy recommended a grant of $218,000 to support a study of encoded reasoning in chain-of-thought models solving math problems. Many researchers are concerned about scenarios where LLM chains of thought might start to hide intermediate reasoning steps, encoding these thoughts in ways that human readers won’t detect. This study, led by Professor Tom Goldstein and Ashwinee Panda, is meant to improve our understanding of the phenomenon of encoded reasoning.
This grant was funded via a request for proposals for projects related to technical AI safety research. This falls within Open Philanthropy’s focus area of potential risks from advanced artificial intelligence.