skipnothing/Regex Engines

Unicode Properties

Type \w+ to grab a word and run it on café. In a browser's regex engine the match stops at caf, dropping the é on the floor, the same silent cut you would get on naïve or Zoë.

  • ▸\p{L} has one answer in every engine that supports it; \w has a different answer in each.
  • ▸\p{L} queries one column, \p{Script=Arabic} another: any letter versus any character of one writing system.
  • ▸\w+ on café stops at caf in JavaScript; the engine isn't broken, \w was defined as [A-Za-z0-9_].
01

The Property Table Behind Every Character

The character A is stored as codepoint U+0041, a number the engine actually sees. That number is a key into a shared data file, the Unicode Character Database, where every assigned codepoint gets one row. A's row records a general category of Lu, an upper-case letter, and a script of Latin. The Arabic م is codepoint U+0645, category Lo, script Arabic; the digit 5 is Nd; the underscore _ is Pc. The engine never inspects the shape of a glyph. It reads the row. Matching a character means asking the table what that codepoint already is.

READING THE ROW, NOT THE GLYPH

Keep going, sign up to unlock the rest

4 more parts in this topic, plus 11+ more topics in Regex Engines.

Sign up, it's freeSee the full Regex Engines