Tokens of (un)appreciation
Do you say please and thank you when talking to AI? Or maybe you get abusive when it starts annoying you?
In my last blog post, we had a look at tokens and the context window, and saw how context can get lost when conversations go on too long.
As we saw in the post, every word that is written, either by you or as a response from the assistant, is using up tokens. And that includes all pleasantries, or even abuse.
So why do it?
It’s very much ingrained in humans to add pleasantries in natural conversations, but is there really any point when talking to an AI assistant? Moreover, is it harmful to your endgoal of getting something done?
The AI doesn’t care, it doesn’t have feelings. If you are using please and thank you, you are slowly using up tokens.
“Thank you so much that was very helpful” - ~8 tokens
“Please continue with what you were explaining” - ~8 tokens
“No, this is not necessary right now please continue” - ~10 tokens
I’m old enough to remember when text messages were charged by the character, we adapted to this by avoiding long words and using abbreviations like BRB instead of typing out I’m a little bit busy right now. I cannot give this all of my attention. I will have to come back to you.
So if I say thank you for your invaluable contribution, you have aided and expanded my knowledge of how everything works and I am gratefully indebted to you…
…I could also just say nothing instead of wasting 20 to 25 tokens
The AI won't feel unappreciated if you skip the thank you, won't be motivated by your gratitude, and won't remember your politeness in the next conversation anyway. (This applies equally to it not caring if you swear at it for suggesting arsenic in a Bolognese recipe.)
These polite expressions are actually taking up token processing time that could be used for more substantive content - it's like using up RAM to run a "politeness program" that serves no functional purpose.
Would you run ls -al please
in a terminal? Well, maybe, if you have a directory called please
and want to list its contents, but other than that you’re just going to get a No such file or directory
error.
AI assistants won’t give you an error, they’ll just ignore it and give you the answer as if you’d never said please.
In ordinary conversation with AI this isn't going to matter too much. But if you're doing anything technical that requires AI generating code or documents then the amount of tokens in the responses are generally larger. At that point it's better to be more direct in your communication to keep noise to a minimum, and it's also in your best interests to tell the assistant to keep responses concise where possible.
Token Costs by Content Type
To illustrate this the table below shows an approximate evaluation of token cost for different scenarios
Content Type | Example | Word Count | Approx Tokens | Notes |
---|---|---|---|---|
Natural Conversation | "Hi, how are you? Could you please help me with..." | 10 | 12-15 | Why? Just why??? |
Direct Request | "Generate Python code to sort a list" | 7 | 8-10 | More efficient than conversational style |
Code Generation | python def sort_list(items): return sorted(items) |
6 | 15-18 | Variable names and syntax each count as tokens |
Code with Comments | python def sort_list(items):\n # Sort the list in ascending order\n return sorted(items) |
11 | 25-30 | Comments increase token usage |
Technical Documentation | "The function accepts an array parameter and returns a sorted version using the built-in sorted() method" | 16 | 20-25 | Detailed explanations use more tokens than code itself |
API Documentation | @param items: List[Any]\n@return: List[Any]\n@raises: TypeError |
8 | 15-20 | Documentation standards add overhead |
Error Messages | "ValueError: Input must be a list or array-like object" | 9 | 12-15 | Technical error details consume similar tokens to natural language |
JSON/Data Structure | {"name": "example", "type": "list", "sorted": true} |
6 | 12-15 | Each key-value pair counts as multiple tokens |
KISS
When you’re doing something technical, keep it simple and you’ll still get the same results
Verbose Example | Efficient Example |
---|---|
"Could you please write a Python function that would help me sort a list of numbers in ascending order? Thank you!" | "Python function: sort list ascending" |
"Could you please create detailed documentation for this function that explains all the parameters and return values? I'd like it to be thorough." | "Document function: params, returns, examples" |
"I'm getting an error with my code and I would really appreciate your help in figuring out what's wrong with it. The error message is..." | "Debug error: [error message]" |
"I would love your assistance in designing a scalable database structure for a social media application. Could you please help me think through the tables and relationships?" | "Design DB schema: social media app" |
"Would you mind helping me understand how to properly structure an API endpoint for user authentication? I want to make sure I'm following best practices." | "API endpoint structure: user auth" |
"Could you please take a look at my implementation and let me know if there are any potential improvements or optimizations that could be made? Here's the code:" | "Review code: [code] optimize performance" |
Does this actually matter?
There could be an argument that practicing politeness is still a good thing in terms of being habit forming which, if broken, could lead to bad communication habits in general, but I’m not convinced. When we talk to our computer, it’s usually to swear at it when it blue screens, and humans muttering under their breath when frustrated hasn’t become an epidemic.
Well, except in the case when you’re waiting ages for an elevator and keep stabbing the button while swearing to yourself.