Initially, for my team, many engineers were worried about using AI in their daily work. They asked some big and reasonable questions: Where do these models come from? Are there any concerns about data breach? And the most difficult thing (it took me a little time to figure it out myself) – am I using it to replace me? As a team member who is actually very passionate about AI, I worry about them.
So I've put together this short and useful manual: It exposes some myths and provides playbooks that will help engineers use AI. For me, AI success is not measured by the number of flashy tools that can spit. That's not the point. The point is, does it actually make us more productive? How effective does it make humans smarter and even more lazy? And I think it's worth breaking things down as engineers are one of the most important consumers of this technology.
So… what is generative AI and what is LLM?
When people talk about tools like ChatGpt, Claude, Amazon Q, and more, what they really mention is Language Model. This is a core technology that enhances much of generative AI. They are trained to understand the written language and respond by generating new text. If these models are expanded to include billions or parameters, then they become what we call them Big language modelor LLMS. LLM is a type of generator AI specially designed to use text and code.
Let's start with the basics.
Not just words, but tokens
In the language model, the smallest unit of understanding is not always the perfect word, and it is token. Think of a token like LEGO Brick: small pieces that come together to build a larger structure. A token can be a complete word (such as “cast”), a portion of a word (“cast”), or even a single letter or character. for example:
Text “I love programming” It may be split into tokens: [I] [love] [pro] [gramming].
This process is called Tokenization. It is how the model learns these materials and builds vocabulary. These vocabulary can be huge. GPT-4 has approximately 100,000 tokens. Claude is even bigger.
Learning language models (different methods)
There are two main learning approaches for language models.
- Mask modelThese models learn by Hide words And you're trying to predict what to do in the blank.
“Tony Stark is also known as the ___ man.”
Models learn to fill in blanks. “Tony Stark” provides the strongest cues, and we can see that “is” and “” are also not worth a lot.
2. Autoregressive model: These models predict Next wordsone word at a time.
First off: “Elsa has entered the castle…”
The model continues: “…and the door was closed behind her.”
In this way, it constructs a story word by word.
Both of these approaches give the model a superpower: Generate text. The term generation AI is the term because a model that can generate open-ended output is called generation.
What does “open-ended output” mean?
Return to the Marvel example:
“Tony Stark is also known as the ___ man.”
Most people (and most models) say “iron.” But models can come up with “strong”, “funny”, or even “family”.
This is an open-ended output. The model is not tied to just one “correct” answer. Make predictions based on probability. Sometimes it nails it. Other times? Not that much. This is why AI can feel both Magic is unpredictable.
Big Leap: From LM to LLM
Scaling up turns a language model into a large language model. It's similar to studying the entire dictionary to get a high score on GRE. Given more data and more parameters, the model can capture more subtle patterns. The parameters are similar to the knobs on the soundboard. The more knobs the more complicated the music becomes. For example, it is estimated that GPT-4 has 1.76 trillion parameters.
Obviously, that requires a huge amount of data. And we can't label data forever. It was self-teacher learning that changed everything. Instead of relying on humans to label every sentence, models can learn on their own by covering words and predicting them. It's like asking blank filling questions at superhuman speed.
And that's why LLMS shines in code: Programming languages are the ideal domain for trainers. The code is very formal and syntactically strict, avoiding the messy whims of natural language. The same actions in Python behave the same way each time, as opposed to English sentences that can hold multiple different meanings depending on the tone, culture, or situation. That accuracy ensures that the model does not need trillions of lines of code to “get it.” Even when the code's dataset is small, the patterns are still consistent enough that the model is well generalized and produces surprisingly robust results.
It's not just LLM
Before we go any further, LLM is one type of generator AI. It just happens to be the most common and most widely used today, but not the whole story.
The current model is normal LMMS (large-scale multimodal model). They don't just use text. It also understands and generates images, audio, and even videos. Therefore, you will see a model that can read blocks of code, explain the image, and translate it into plain English at once.
Over the past few years, there have also been other terms in conversation. Basic model. These are very large models trained on a wide range of general purpose data. Think of them as “first floor” or “vanilla flavor of the ice cream world.” Once trained, you can then tweak or stretch to apply to more specialized tasks, such as creating software documentation, powering company chatbots, and reviewing contracts. “Foundation” is a term used to describe them in opposition to smaller specialized models developed over them.
General myths (if you don't care about definitions or details)
Myth 1: “AI understands like we do.”
fact: AI does not “understand” in the human sense. Previous texts (contexts) make educated speculations about what comes next. A good prediction can simulate understanding, but it is far below the hood probability. Nevertheless, some experiments are intriguing. One blog showed that different models appear to play “personalities.” Openai's O3 turns out to be a master of deception in a diplomatic game, but Claude Opus 4 wanted everyone to get along.
Myth 2: “AI just copy training data.”
fact: Does not save examples of training like clipboards. Save the pattern to the weights (number dials in the model) you want to adjust during training. When you query, the model is not a copy pasty, but instead creates a new sequence. That being said, data privacy remains an issue. Even when private data is used for training, it can still pop out indirectly or be read by humans during reinforcement.
Myth 3: “The prompt is hacking.”
fact: The prompt is not a hack, it is the steering wheel. The prompt sets up the context, so the model knows which tokens weight more heavily. If you still remember the previous example:
“Tony Stark is also known as the ___ man.”
Here, “Tony Stark” has the largest weight and the capital “Man” helps to zero the model into “iron”. The filler word (“if it is also known”) weighs relatively little. An effective prompt does the same thing. Emphasises important cues so that the model can reach the correct inference. It's not a hack, it's guidance.
Myth 4: “AI always give the same answer.”
fact: no. Generation AI relies on probability rather than pre-programmed scripts. That is, the same question has different answers depending on randomness, temperature settings, or slight differences in prompts. Sometimes it's great (brainstorming), sometimes confused and frustrating (inconsistent answer).
Myth 5: “The bigger models are always better.”
fact: Size helps, but that's not all. In other words, big isn't necessarily good. Huge models like the GPT-4 may be surprising, but smaller specialized models can outweigh that in certain tasks. Agility and specialization can sometimes defeat power.
Myth 6: “AI can replace engineers.”
fact: Generic AI is a great programming assistant thanks to its systematic and temperament towards the programming language. However, it is not yet qualified as a lead engineer. Rather, it's similar to a speedy internship. Speedy internships are great for creating rough drafts, boilerplates and even great snippets, but require review and monitoring.
in short, Generator AI does not replace engineers, but it changes what engineering is. The skill sets vary. Instead of spending most of the time entering every line of code, engineers can focus more on system architecture, design, and project monitoring. Generic AI can write features, but it cannot measure trade-offs, understand business goals, or manage complexity at scale. That's where people need to be at the helm.
Myth 7: “AI is too stupid to make things happen.”
fact: It is usually hallucination that AI appears to be extremely stupid. Remember when the model told you that it provided an open-ended response based on probability? That's where it begins. If your model has no context, or if the fact you are asking is unusual, you still need to fill in the blanks and make yourself happy. So, while that may be wrong, you're anticipating something that sounds good.
Another common reason is: Context window. The model only has the opportunity to “see” many tokens at once, like short-term memory buffers. If your question depends on information outside that window, it will simply be lost. The model then guesses and fills in the blanks and can turn off completely.
Hallucinations do not mean that AI is malfunctioning, but are the result of a system designed to be freely generated and not follow the script.
Practical Playbook: Using AI as a Software Engineer
Think of AI as a (human) team member. Tell us what you need in plain language. Don't worry about grammar and don't try to make it interesting. Keep being direct. Ask as many “silly” questions as you need along the way. The loop is simple: questions, reviews (results), improvements (requests), repeat.
1) First Context
Tell the model who this is for, what the goal is, what constraints are important.
Examples of mini prompts:
You are helping our payments team. Goal: add idempotency to the charge endpoint.
Constraints: Python 3.11, FastAPI, Postgres, existing retry logic.
Output: explain first, then give code in ```python``` fences.2) Decide the task
“Complete” is clearly defined to clearly define procedures or acceptance criteria.
Examples of mini prompts:
Make a step list for the refactor. Include pre-checks, code changes, tests, and a final rollout check.
Mark each step with [owner], [est], and [risk].3) Constrain the output
Find the shape so that you don't get a wall of text.
Examples of mini prompts:
# Example 1
Return JSON with keys: rationale, risks, code, tests. No extra prose.
# Example 2
Generate a SQL query inside sql … fences.4) Check and repeat
Treat the first draft as a sketch. Please point to the wrong work. Please ask for a correction. Loop again.
5) Keep live log files
Save each large step and ask the model to change it to a file so that the context can be reused later.
Mini Prompt example:
Start or update a file named AI_LOG.md.
Append a dated entry with sections: Context, Decision, Commands, Snippets, Open Questions.
Only add new content. Keep older entries.6) Work within the context window
The model only “sees” short snippets of recent text. If the thread is too long, it compresses the chat history.
A. Do it at the prompt. You can use the following compression prompt to paste when the chat is heavy:
Summarize all prior messages into a compact brief I can reuse.
Format:
- Facts we agreed on
- Constraints and conventions
- Decisions and their reasons
- Open questions
- Next actions
Keep it under 300 tokens. Preserve file names, APIs, and versions exactly.B. Use the tool if available. Some services employing AI include tools to support them. For example, Amazon Q has a /compact A command that condenses chat history into short ones. Warns when the conversation is approaching its limits, suggesting compression. (Related AWS Documents)
C. I re-identified the next turn. Paste the back after compression as a new context. Next, the next request.
I'll summarize
Generator AI is not a replacement for engineers, but it is revolutionizing what engineering is. It's well-utilized, and not your competitor, an assistant that helps you speed up your dragging, allowing you to focus on design, systems and difficult decisions. A thriving engineer is someone who can guide it, push it back and turn it into a team player. Because in the end, even Iron Man had to fly his suit.
Did you enjoy this? For more information about TDS, please subscribe to my newsletter for more information. thank you!
