Max New Tokens

The maximum number of tokens to generate, exluding the prompt. Use this parameter to limits the number of tokens to generate. In other words, max new tokens lets you control how long the AI’s response will be. Think of it as setting a word limit for an essay.

If you want shorter, more concise answers, set a lower number of max new tokens. This tells the AI to keep its answers brief. This also decreases the response time.

For more detailed responses, increase the max new tokens. This tell the AI to fully describe something.