15 September 2026 · Syamjith NK
Extract the text from a PDF invoice written in Arabic with pypdf, and until yesterday every Arabic-Indic number on the page came back reversed. A total of ١٢٣٤ extracted as ٤٣٢١. No error, no warning, and a number that still looks entirely like a number.
I filed the fix on 13 September and it was merged on the 14th. The library change is two lines. The other 81 lines of the patch are the test. What is worth writing down is not the patch, it is why this particular mistake is so easy to make that a library with ten thousand stars carried it.
pypdf has to decide, character by character, whether text is running left to right or right to left, because a PDF content stream stores glyphs in the order they were painted and not in reading order. Its rule is a list of codepoint ranges. Anything in the right-to-left list gets prepended to the string being built up rather than appended, which is what turns painted order back into logical order.
The Arabic entry in that list is one broad block:
RTL_CHARACTER_RANGES = (
("\u0590", "\u08FF"), # Hebrew, Arabic, Syriac, Thaana, N'Ko, etc.
...
)That range is correct for the letters. It is also, unavoidably, where the Arabic-Indic digits live: U+0660 to U+0669 sits in the middle of it. So each digit of a number was prepended in turn, and a four-digit number came out backwards.
Arabic script runs right to left. Arabic-Indic numbers do not. They are written and read left to right, most significant digit first, exactly like Western digits — which is unsurprising once you remember that Western digits are themselves a borrowing of this notation.
Unicode encodes this directly. A character does not have a script and inherit its direction
from it; it has its own bidirectional class. Arabic letters are class AL, Arabic
letter. Hebrew letters are R. The Arabic-Indic digits are class AN,
Arabic Number, and the bidirectional algorithm does not reverse a run of them.
The Persian digits make the point even more bluntly. They are not AN; they are
class EN, European Number — a name that is odd until you accept
that these classes describe numeric behaviour and have nothing to do with which script or region
the character comes from. You can check any of this in one line with
unicodedata.bidirectional():
| characters | bidi class | reversed on extraction? |
|---|---|---|
| Arabic letters | AL — Arabic Letter | yes — correctly |
| Hebrew letters | R — Right-to-Left | yes — correctly |
| Arabic-Indic digits U+0660–U+0669 | AN — Arabic Number | no |
| Persian digits U+06F0–U+06F9 | EN — European Number | no |
So the defect is one substitution: treating this character belongs to an Arabic block as this character is right-to-left. For letters the two are the same thing, which is exactly why the shortcut survives. It only comes apart on the digits, and the digits are the part of an Arabic document a developer is least likely to be reading.
This is the same family of error as the one I wrote about in the reshape+bidi recipe: direction and ordering are properties of individual characters, and every bug in this area comes from applying a decision to a whole string that only holds for part of it.
Reversed Arabic prose announces itself. مرحبا coming out as ابحرم is not a word; anyone who reads the script sees it instantly, and even someone who does not can usually tell the shapes are wrong.
٤٣٢١ is a perfectly good number. It is the wrong one, but nothing about it looks damaged, and there is no proofreading pass that catches it. It is the same property that makes the lam-alef case unfixable after the fact: corruption that lands on a plausible value is far more dangerous than corruption that lands on nonsense, because the plausible one gets believed.
The class of document where this matters is not exotic — invoices, bank statements, dates, identity numbers, any Arabic PDF that a pipeline reads rather than a person. I have not measured how much of that exists or how much of it broke, and I am not going to guess. The mechanism is enough of a reason to fix it.
pypdf already has a second list, NEUTRAL_CHARACTER_RANGES, for characters that
belong to neither direction and should simply be appended: ASCII punctuation, general punctuation,
currency symbols. The digits belong there. That is the whole change:
NEUTRAL_CHARACTER_RANGES = (
("\x00", "\x2F"), # ASCII control codes, space, and early punctuation (!"#$%)
("\x3A", "\x40"), # ASCII operators and punctuation between digits & A (:;<=>?@)
+ ("\u0660", "\u066D"), # Arabic-Indic digits and Arabic numeric separators
+ ("\u06F0", "\u06F9"), # Extended Arabic-Indic (Persian) digits
("\u2000", "\u206F"), # General punctuation
("\u20A0", "\u21FF"), # Currency symbols, ...
)The first range runs to U+066D rather than U+0669 because the characters just above the digits are the Arabic percent sign, decimal separator and thousands separator — punctuation that appears inside a number. Moving the digits out while leaving those behind in the right-to-left list would have broken ٥٠٪ in a new way. U+066D itself, the Arabic five-pointed star, is not numeric at all; it is just where a contiguous range has to end.
Testing PDF text extraction usually means committing a PDF, and a binary fixture is a thing nobody can review. This test builds its page instead: a Type0 font with Identity-H encoding, where the character codes are just 1, 2, 3… and a ToUnicode CMap maps them back to the characters under test. There is no embedded font program, so what comes out depends only on pypdf's own handling, and the whole fixture is readable Python.
Five cases run against it, and the two that matter most are the ones where the expected output is unchanged:
| painted on the page | pypdf extracts | what the case pins |
|---|---|---|
| ١٢٣٤ | ١٢٣٤ | Arabic-Indic digits keep their order |
| ۱۲۳۴ | ۱۲۳۴ | Persian digits likewise |
| ٥٠٪ | ٥٠٪ | digits with the Arabic percent sign |
| ابحرم | مرحبا | Arabic letters are still reversed |
| םולש | שלום | Hebrew is unaffected |
Widening a neutral list is the kind of change that fixes one thing by breaking another. The obvious failure mode here is over-correcting: pull too much out of the right-to-left range and Arabic words stop being reordered at all, which trades a wrong number for wrong prose on every page. The last two cases are there to fail if that happens. A test that only proves the bug is gone does not tell you what the fix cost.
It is two lines in a library I had not contributed to before, merged by
stefan6419846 the day after it was opened. It is
not in a released version: the latest tag, 6.18.1, predates the merge by three days, so
pip install pypdf today still reverses your digits. It does not close
#1629, the broader issue about Arabic
mixed with digits and punctuation, which has been open since February 2023 and remains open. And
one merged pull request does not make me a pypdf contributor in any sense worth claiming.
What it is, is a small confirmation of something the last few weeks of this have kept demonstrating: the bugs in Arabic text handling are rarely hard, they are unattended. Two lines, in plain sight, in a repository with ten thousand stars and an open issue touching the same area since 2023. They survive because almost nobody who reads the code reads the output.