How LLMs Actually Learn: Fine-Tuning, Transformers, and the Architecture Behind Modern AI

Modern AI models like GPT, Llama, and Claude aren't built from scratch every time you want them to do something new. They're shaped — first through massive pre-training, then refined through a process called fine-tuning. This article breaks down how that works, what's actually happening inside the model, and why the Transformer architecture made it all possible.

Fine-tuning: continuing the learning process

Fine-tuning is essentially continuing the training process on a smaller, targeted dataset.

The model already has weights (billions of numerical parameters) from pre-training on vast amounts of text. Fine-tuning adjusts those weights further using your specific dataset. You run forward passes (predictions) and backward passes (error correction) just like original training, but on a much smaller scale.

Fine-tuning cost varies a lot depending on model size and whether you use a cloud API (e.g. OpenAI's fine-tuning endpoint) or run it yourself on rented GPUs. RAG ongoing cost depends on embedding and retrieval infrastructure, but a basic setup is essentially free.

What the model does with the data

It sees input/output pairs repeatedly and nudges its weights to make those outputs more probable given those inputs. If your dataset has thousands of examples of "question → helpful answer", the model gradually shifts its probability distributions to favour that style of response. It's not memorising the data like a lookup table — it's shifting patterns across the whole network.

Types of fine-tuning

• Full fine-tuning — update all the weights. Expensive, and risks overwriting general knowledge ("catastrophic forgetting").
• PEFT (Parameter-Efficient Fine-Tuning) — the modern standard. Rather than updating all billions of weights, PEFT methods freeze most of the model and train only a small subset of parameters. This dramatically cuts compute cost while preserving the base model's general knowledge.
  • LoRA / QLoRA — the most popular PEFT approach. Inserts small trainable adapter matrices alongside the frozen weights. QLoRA adds quantisation so it runs on consumer hardware.
  • Prefix tuning / Prompt tuning — prepends trainable tokens to the input rather than modifying weights at all. Even lighter than LoRA.
  • Adapter layers — inserts small trainable modules between transformer layers.
• Instruction tuning — teaching the model to follow instructions, using datasets of prompt/response pairs. Often combined with a PEFT method.
• RLHF — uses human preference ratings to steer the model toward better outputs (how ChatGPT was shaped).

Practical reality

Dataset quality matters far more than size. A few thousand clean, well-formatted examples often beats millions of noisy ones. The model is also very sensitive to format — if your training data uses a specific prompt template, you need to use that same template at inference time.

For open source models like Llama or Mistral, tools like Hugging Face's transformers + trl library, or Unsloth for efficiency, handle most of the heavy lifting.

Q&A: common questions

Is the structure graph-like?

Not exactly. Transformers are stacked layers — embeddings → attention → feed-forward → output. The attention mechanism feels graph-like (each token connects to every other token) but it's recomputed fresh each pass, not a persistent structure. Underneath it's all matrix multiplications.

Can you train a model only on Python for banking?

Totally doable. You'd fine-tune a base model (which already knows Python) on banking-specific code — risk calculations, transaction processing, regulatory reporting, internal conventions, etc.

The main risk is over-specialisation, where the model gets good at banking Python but loses general ability. LoRA fine-tuning helps since it preserves the base weights. It's also worth considering RAG as an alternative — feed the model relevant code and docs at query time rather than retraining at all. Simpler to maintain.

What is a Transformer?

The Transformer is the architecture underpinning virtually every modern LLM (GPT, Llama, Mistral, Claude, etc.). It was introduced in the 2017 paper "Attention Is All You Need".

Before Transformers, models processed text sequentially — word by word. Transformers process the entire sequence at once and learn which parts of the input to pay attention to when predicting each output.

Key components

• Tokenisation — text is split into tokens (roughly word fragments) and converted to numbers.
• Embeddings — each token becomes a vector representing its meaning in high-dimensional space. Similar words end up close together.
• Attention mechanism — the key innovation. For each token, the model asks "which other tokens in this sequence are relevant to understanding this one?" It computes a weighted relationship between every token pair — this is how it understands that "it" in "the bank collapsed because it was unstable" refers to "bank", not "collapsed".
• Feed-forward layers — after attention, each token passes through a small neural network that transforms it further.
• Stacking — these attention + feed-forward blocks are stacked many times (GPT-4 likely has 96+ layers). Deeper layers capture increasingly abstract concepts.
• Output — the final layer produces a probability distribution over the vocabulary: essentially "what token is most likely to come next?"

Why it was a breakthrough

• Parallelises well across GPUs, unlike sequential models
• Captures long-range dependencies — it can relate a word at position 1 to a word at position 500
• Scales — more data and compute keeps improving performance, which wasn't true of earlier architectures

Summary

Fine-tuning lets you take a powerful pre-trained model and specialise it for your use case without starting from scratch. The Transformer architecture, built on attention mechanisms and stacked layers is what makes modern LLMs capable of understanding context at scale. Whether you're building a banking code assistant or a customer support bot, the workflow is the same: start with a strong base model, fine-tune on high-quality targeted data (ideally with LoRA to preserve general knowledge), and consider RAG when the domain knowledge changes frequently.