Amazon has warned its software engineers against treating AI token consumption or generated code as proof of progress, describing those measures as “deeply flawed” in guidance on the role of developers and managers. The memo, circulated internally in September and posted to the company’s website on Tuesday, sets out Amazon’s expectations for staff using AI and emphasizes results over the volume of activity.
The guidance addresses “tokenmaxxing”: using AI extensively with little regard for the usefulness of its output. Amazon argues in the memo that turning activity measures into targets can encourage higher spending and unnecessary complexity. It also says engineers should recognize when AI is unsuitable for a particular task.
An Amazon spokesperson said the company had not instituted caps on employees’ token usage. The memo’s criticism of usage-based measures does not amount to an instruction to disregard that data altogether.
Assessing results alongside AI usage
The memo identifies token consumption, adoption of AI tools, and the number of code changes or reviews as flawed ways to assess teams or organizations when disconnected from outcomes. Amazon says those measures require careful interpretation alongside other information about performance.
Amazon’s stated preference is to judge work by long-term results. The memo also acknowledges the difficulties with that approach: outcomes can take time to appear, be hard to measure and be difficult to separate from other trends. It presents usage data as one part of an assessment, with judgment needed to connect activity to results.
Marc Brooker, a vice president and distinguished engineer at Amazon Web Services, said Amazon has looked at “feature velocity” as a way to assess whether AI is improving engineers’ productivity. He described it as examining “how many launches are we doing, how many successful changes are we making in production”.
“Broadly, we think about this as a return on investment,” Brooker said. “If there is an important problem to solve for our customers, important problem to solve for our business, we’re going to invest tools, time, tokens, human effort, etc. into those.”
Brooker also acknowledged that experimentation remains necessary because the value of a problem is not always clear in advance.
“What we don’t want to see is wasted effort where we are going and spending those tokens solving problems that don’t matter and sometimes it’s hard to know, so we do need to have some appetite for experimentation,” he said.
Recognizing where AI tools fall short
The memo separately addresses engineers whose work does not benefit from the tools available to them. Amazon frames the ability to explain those limitations as part of the judgment it expects from staff.
“If your work does not benefit from current AI tooling, that is a finding, not a failing,” Amazon wrote.
The document asks engineers to record why the tools are not helping, share that experience with peers, discuss it with their manager and work toward solutions. Its guidance treats identifying limitations as information that can help shape engineering practice.
For technical and organizational leaders, the memo calls for direct experience with AI tools in production work and an understanding of how others use them. Leaders should be able to describe a task where the tools helped, another where they did not, and explain their reasoning to their teams, it says.
Amazon also asks leaders to balance exploration of new methods with investment in approaches that already work. Those expectations accompany the memo’s focus on improving delivery for customers and the business.
“An engineer who can explain where AI does not yet help their work is demonstrating the judgement we are asking for,” Amazon wrote.
