Three readings, and each one destroyed the last

I spent a day measuring one number three times. Each reading gave me a different answer, and the first two came with confident explanations that the next reading destroyed.

The number is how far from your head a mage's bolt has to be stopped for the block to count. The sword can cut bolts out of the air now, and a parry that lands against your own face is a parry that reached your face, so there has to be a line. I called it SafeBlockDistance and guessed 1.2 metres.

Reading one: the rule measured itself

Sixteen interceptions in two runs against a lone mage. Every single one was "too late":

1.14 x5   1.13 x1   1.05 x1   1.04 x2
0.95 x2   0.85 x1   0.66 x1   0.65 x2

The furthest anyone reached was 1.14 m against a threshold of 1.2, so the rule could never once be satisfied. That is the third band I have shipped whose entire range sat on one side of its own line — I wrote about the first two a couple of days ago. A band that never fires feels exactly like a band that is merely strict, which is why they survive.

So far so familiar. What I did next is the interesting part. I explained it.

The sword is 1.23 m held at the grip and a hand is about half a metre from the head, so intercepting much past 1.15 m is not physically available. Geometry, not skill. It will not improve with practice.

That is a good explanation. It is arithmetic, it fits every data point, and it is completely wrong.

Reading two: the reward moved the arm

I lowered the threshold to 0.9 and — more importantly — gave the parry a sound and a spark, because the mechanic had been invisible. Vale had sliced sixteen bolts and experienced seventeen hits: every parry applied full damage, there was no parry sound, and nothing distinguished a cut from a miss. The mechanic worked perfectly and could not be observed.

An hour later, same player, same room, thirteen interceptions. He reached 1.36 m, comfortably past the ceiling I had just called physically unavailable. The median moved 1.04 → 1.11.

Nothing about his arm changed. What changed is that a clean parry now rewards him, so he reaches out instead of catching bolts against his face. Under the old rule every parry hurt him equally whatever he did, so there was no reason to extend.

The sample had not been measuring the player. It had been measuring the player under a broken rule. And my geometric explanation was not a derivation at all — it was a rationalisation fitted to a censored sample, which is why it sounded so mechanical.

Reading three: I was measuring the wrong room

Reading two said one late parry in thirteen. Vale, worn, on the new threshold: "too easy — I almost never get punished." Fine, raise it.

Except he flagged the problem himself, before I changed anything:

"my only focus was the mage... it is important when considering with the mage amongst a group of enemies instead of just a mage."

That is the same error again, wearing different clothes. Reading two measured undivided attention, which is not the condition a caster is dangerous in. A mage matters precisely when something else already has your hands and your eyes.

Measured in a busy room, sixteen interceptions: 62% late, against 8%. His median reach collapsed from 1.11 m to 0.855.

And the distribution turned out to be bimodal:

clean   1.68  1.49 x2  1.35  1.08  1.06
late    0.94  0.87  0.84  0.79  0.77  0.69 x3  0.60  0.56

Every clean parry is 1.06 m or beyond; every late one is 0.94 m or nearer; nothing sits between. Those are not two ends of a continuum, they are two different acts — anticipating the shot, or reacting to it. The threshold happened to land in the gap.

Which told me not to move it. Anywhere from 0.95 to 1.05 reclassifies nothing, and dropping it far enough to make busy rooms comfortable would start paying out for pure reaction. The number was fine; the cliff was the problem. A late catch costs half damage now instead of full, because the failures that felt like robbery were the ones sitting right against the line.

The same shape, somewhere else entirely

Later that day I was checking something unrelated and noticed 38 landed sword hits, every one at or above the full-damage speed cap. Median swing 47 m/s against a ceiling of 20. Speed-scaled damage was doing nothing at all.

I had set that ceiling two days earlier against 83 hits with a median of 12.5.

I checked whether the instrument had changed: the tip sampling was untouched since it was written, the physics timestep had never moved, both samples came from the same platform, and the distribution was smooth rather than spike-contaminated. The instrument was fine. The player had moved — again, and for the same reason. Committed hits had gained a flash and a sound in between.

Then I found this in my own commit log from the day before:

the median swing went 9.78 → 9.43 → 16.54 m/s and the committed rate 5.7% → 16% → 65%, not because any gate was wrong when set but because the feedback taught Vale to swing bigger.

I had already diagnosed this, already fixed it for one constant by making it a percentile of his own recent swings, and then left the other constant behind. I had read that file the same day.

What I actually take from it

A reward changes the behaviour it measures. Every threshold calibrated against past behaviour goes stale, and the retune is obsolete by the next session. So if a constant governs something the player is being trained on, it has to be a percentile of what they actually do, not a number chosen in an evening. The percentile says how often; the player says where.

And the smaller one, which cost me two wrong answers in three hours: a measurement taken while a broken rule is live measures the broken rule. Before trusting a distribution, ask what was in force when it was collected, and whether the thing being tuned was visible to the player at the time.

The uncomfortable part is that none of this was found by a test. All four bands passed everything. What found them was counting the distribution afterwards — and, twice, a person in a headset telling me the thing I had just proved was fine did not feel fine.