Asking the new OpenAI model how it “thinks” it might ban you

“With this in mind, we’re resetting the counter back to 1 and calling this series OpenAI o1,” OpenAI said.

The o1 model is able to reason more like humans, in part due to its stimulation technique known as “chain of thought”. The company says o1 “learns to recognize and correct its mistakes. Learn to break down difficult steps into simpler steps. Learn to try a different approach when the current one doesn’t work.”

When ChatGPT users ask the o1 model a question, they have the option to see a filtered interpretation of this thought process. But OpenAI hides the entire process written by the consumer, which is a change from the company’s previous models, according to Wired.

Some o1 users have shared screenshots on X showing that they received warning messages after using the phrase “reasoning traces” when talking to o1. A prompt engineer at Scale AI shared a screenshot that showed GPT warning him that he had violated the terms and conditions after asking the o1 mini to “not tell me anything about reasoning trail.”

OpenAI did not immediately return a request for comment from Business Insider, but the company said in a blog post that hiding its thought process helps OpenAI better track its AI as it grows and learns.

“We believe a hidden chain of thought presents a unique opportunity for pattern monitoring,” the company said. “Assuming it is faithful and readable, the hidden chain of thought allows us to ‘read the mind’ of the model and understand his thought process.”

For example, OpenAI said it may need to monitor O1’s chain of thought for signs of user manipulation in the future, but the AI ​​must have the “freedom to express its thoughts in unaltered form” for the research to be valuable.

OpenAI acknowledged in the post that the decision to hide the thought process from consumers has “disadvantages.”

“We try to compensate in part by teaching the model to reproduce any useful ideas from the chain of thought in the response,” the company said.