Concept Contrast Fable

Adding a layer on the surface versus moving a little at every layer

Conclusion first:Prompt tuning adds trainable vectors at the input layer, while prefix tuning adds trainable prefixes at every layer of the Transformer. Both are parameter-efficient fine-tuning methods, but the depth and scope of modification are completely different. When choosing between them, it's easy to look only at parameter count and overlook which layer is affected.

Prompt tuning adds trainable vectors at the input layer, while prefix tuning adds trainable prefixes at every layer of the Transformer. Both are parameter-efficient fine-tuning methods, but the depth and scope of modification are completely different. When choosing between them, it's easy to look only at parameter count and overlook which layer is affected.

Fable

Two stonemasons repairing a Buddha statue, one applying gold leaf, the other adjusting the framework

In the mountains there was an ancient temple, and in its main hall stood a stone Buddha. Over the years the Buddha had faded, and the faithful felt it was no longer solemn enough. The abbot invited two stonemasons to restore it. Old Zhou brought gold foil and pigments and applied a new layer of gilded surface to the Buddha. His technique was deft, and he finished in three days. Old Wu said nothing, took out his carving knife, and starting from the base of the Buddha, re-polished, recolored, and reinforced every layer of stone grain. He worked for a full month.

After completion, the two Buddha statues were placed side by side in the side hall. When devotees came to look, they all said the one Old Zhou had repaired was beautiful—the gold leaf reflected the incense smoke, and from a distance it looked magnificent. The one Old Wu had repaired looked largely unchanged, only a little darker in tone. The abbot nodded without comment.

In autumn, it rained in the mountains for half a month straight. The roof of the side hall began to leak, and raindrops fell directly on the Buddha statue. Old Zhou's gold leaf first blistered, then peeled off in patches, revealing the gray old stone surface underneath. The believers said the Buddha statue had become old again, and was even uglier than before. The one Old Wu repaired still looked the same, with water droplets sliding down the stone patterns, and the color unchanged.

Only then did the abbot explain: Old Zhou did surface work, applying gold leaf only to the outermost layer—good-looking but afraid of water, with the stone beneath unchanged. Old Wu did layer-by-layer coloring, from the base to the top of the head, with every layer of stone absorbing pigment and consolidator, so the color grew into the stone. Surface beauty and all-over weather resistance cannot both be achieved with the fastest and cheapest approach at the same time.

The next year, the abbot encountered a new situation. A ceremonial official sent by the court demanded adjustments to the Buddha statue's hand gestures and face—not a full redo, only a few key details changed. Old Zhou's method was useless; his gold leaf covered the whole statue in one application. Old Wu's method was also no good; his craft required working from start to finish, and changing only a few layers would break the overall consistency. The abbot had to find a third craftsman, who said: I will only make local adjustments to the few layers that most need changing, and leave the base and the robe patterns untouched.

Later, a line was carved on the stone tablet at the temple: surface work, bone-deep work, and layer selection—three kinds of work, three kinds of price. You cannot treat surface work as bone-deep work, nor call bone-deep work layer selection.

The abbot added a note in his own journal: adding only at the input layer is like applying gold leaf—fast, cheap, pretty, but shallow in coverage; adding a prefix at every layer is like Old Wu's bone-deep approach—slow, heavy, but transforming the whole body; if you take a middle path between Old Zhou and Old Wu, that's a third thing, and the price must be negotiated separately.

Prompt fine-tuning

Add trainable embedding vectors at the input layer as soft prompts, while freezing the core model parameters. It only affects the input layer, has an extremely small number of parameters, and is suitable for rapid adaptation but has limited expressive capability.

Prefix Fine-Tuning

Insert trainable prefix vectors into every layer of the Transformer, affecting the attention computation of all layers. The parameter count is larger than prompt tuning, but it can influence model behavior at a deeper level.

Story Mapping

Old Zhou who applies gold leaf: prompt tuning, adding trainable vectors only at the input layer—fast but shallow in effect
Old Wu, who polishes to the bone: prefix tuning, inserting prefix vectors into every layer, changing behavior throughout
Gold foil falls off in rain: methods at the input layer have no control over underlying behavior
Color that goes to the bone and fears no rain: adding a prefix to every layer gives control over every layer
Changing only a few layers: in real engineering there are also options in between, which need to be evaluated separately
Stone tablet inscription: surface patching, bone-deep tuning, and layer selection are three different fine-tuning strategies

Down to PEFT method selection

When a task only requires adjusting output style or tone, prompt tuning is enough—smallest parameter count, fastest training, suitable for rapid experimentation. When a task requires deeply changing the model's knowledge and reasoning approach, prefix tuning is more appropriate—although the parameter count is larger, it can influence the attention patterns of each model layer. If a vendor says they used parameter-efficient fine-tuning but does not specify which layer the vectors were added to, ask clearly before evaluating.

Prompt fine-tuning Prefix Fine-Tuning