Training, Reading, and Editing Legible Transformers

Legible-by-Construction Transformers utilize named fuzzy set operations to replace opaque dense activations. By applying a sigmoid to attention head values, channels become readable feature detectors without increasing parameters. While a crispness penalty aims to sharpen these operators, it can cause collapse into dead constants. To resolve this, a per-channel variance floor is introduced as a loss metric. This ensures bounded units remain decisive detectors, maintaining model quality while making both feed-forward and attention layers fully interpretable.