7 September 2026 · Syamjith NK
Once you know that get_display(reshape(text)) corrupts Arabic on a stack that already shapes, the next thought is obvious and it is the one I had: fine, I will just undo it.
The corruption is mechanical. Reshaping maps letters to Unicode's Arabic Presentation Forms; the bidi pass reverses the order. Both look invertible. Normalise the presentation forms back with NFKC, reverse the string, done.
It round-trips exactly. I tested it on a lot of text and it round-trips exactly — until the text contains a lam-alef.
Arabic has four lam-alef ligatures — لا لأ لإ لآ — where the letter lam followed by alef is encoded as a single codepoint rather than two.
That single codepoint is what breaks the inverse. NFKC expands it back into its two constituent letters in logical order, while every character around it is still sitting in visual order from the reversal. So the pair comes out correct with respect to itself and backwards with respect to its neighbours.
The result is not garbage. That is the whole problem.
| original | after the naive fix | |
|---|---|---|
| الإمارات | اإلمارات | not a word — someone will notice |
| السلام | السالم | a real word. A different one. |
The second row is the dangerous one. السلام is "the peace". السالم is a real, well-formed Arabic word, and it is not that. It passes a spell check. It survives a human proofread, because a proofreader reads for sense and it makes a kind of sense. And it silently breaks anything downstream that does exact matching — search, deduplication, compliance scans, a lookup against a list of names.
This is not a corner case. ال is the definite article; alef is the commonest letter in the language. "Definite article followed by alef" is one of the most frequent sequences in written Arabic. Any real corpus is full of it.
I wrote arabic-lint to find this class of corruption — Arabic that was mangled before it was stored, written back into JSON, localisation files, CSV exports and source, where every later render faithfully reproduces the damage.
pip install arabic-lint
arabic-lint ./locales
Zero dependencies, exits non-zero on findings, drops into CI. And it deliberately has no --fix.
That was a choice, not an omission. A fixer that is correct 95% of the time and silently produces a plausible wrong word the other 5% is worse than no fixer, because it converts a visible problem into an invisible one. The only safe repair is to go back to the source that was corrupted and re-export it.
The detector looks for Presentation Forms-B characters in stored text, on the reasoning that typed Arabic never uses those legacy compatibility ranges. My unit tests all passed.
Then I ran it across 3,826 real files and it found two categories of false positive I would never have invented:
Both are now regression tests. The general lesson has cost me more than once: a detector that has only ever seen data you wrote is a detector you have not tested. Point it at a large pile of real files before you believe it.