My biggest issue with regular expressions is remembering the exact syntax of each of the different common regex engines. Javascript, Perl, `grep`, `grep -E`, vim, `awk`, `sed`, etc...
Each one seems to have slightly different syntax, require different characters to be escaped, has different defaults (is global search enabled by default? Multiline? What about case sensitivity?), some don't support certain lookarounds, how does grouping work, and so on.
It's a real pain, especially when you want a quick one-off regex. In that case the learning curve changes the economics of what the right tool is for the job. Often I'll just end up writing a program using a tool I already know, even though I'm aware that it's a less efficient choice—at least its inefficiency is predictable. Of course if you do that enough times then you've spent more than the original learning curve would have cost! I have done this in painfully many contexts. Regexes are an obvious case, probably because they're so obviously doing the same thing, just differently enough to waste your time.
Some people actually like searching for and paging through documentation to learn how, e.g., regex format X does character escaping. And they tend to remember such things, too. I don't and don't.
This is a very big issue and one I recently faced in the workplace. Needed a script to parse some log files to generate CSV reports for the business users. I knew jq (https://stedolan.github.io/jq/) and hence was able to write less than 10 lines of jq to do it combined with some preprocessing using sed.
I then realised that NOBODY on the team knew jq apart from me and I had to rewrite it in Python which took me 4 days to do correctly and handle everything that jq did for me.
There are three types of regex as far as I know: basic (aka GNU), extended and PERL. Grep uses GNU as the name implies.
egrep or grep -E uses extended. PERL is used elsewhere like JavaScript. Typically you'll see pcre which is the library for Perl Compatible Regular Expressions.
The name "grep" is unrelated to GNU. According to The Jargon File, the etymology is:
from the qed/ed editor idiom g/re/p, where re stands for a regular expression, to Globally search for the Regular Expression and Print the lines containing matches to it
In fact, grep predates the GNU project by almost a decade.
Also, it's the first time I hear about "GNU" regexps.
There are many differencese between "compatible" regular expressions, for (many) examples read the end of section "Important Notes About Lookbehind" on [1] or compare 24 dialects on [2] for example.
Erlang requires doubling up on escape backslashes, which generally causes me grief even after I flail about and finally remember it. Subtleties abound.
I wonder if there is a tool that lets you enter a regular expression from one engine and get back the equivalent regular expressions for other engines?
Some of the look arounds might be too powerful to do in some engines so it would not always work, but it would still be quite useful if it just handled the differences in escaping, specifying modes like case insensitivity or multiline matching, and referencing match groups in replacements.
Not exactly, but that's a(n internal) feature of my Strukt[1]. It's how the optimizer is able to take any of {exact string, glob, ICU regex} as user input, and convert them into efficient queries for various data sources.
For example, if you do ListFiles[folder=~/Downloads] -> FilterString[field=filename, regex=\.pdf$], it will parse the regex, verify that it only uses features which are available in Spotlight's query syntax (roughly, a glob with slightly funny syntax), and rewrite that adjacent pair of operations into a single API call behind the scenes:
kMDItemFSName LIKE[c] "*.pdf"
I can report, having written parsers for a few common regex dialects, that there are a ton of obscure regex features, with different semantics everywhere. 100% conversion is almost never possible.
A lot of the work in deciding if a translation is possible is in identifying if something is good enough, e.g., SQLite can't do case-insensitive search (in general), but if your regex happens to be /([0-9]+)/ then case-sensitive search will work just fine. Fortunately, for Strukt, if a conversion is impossible, I can just run the operations as written: it's much slower, but still correct.
I've thought about breaking this part of Strukt off into a tool just for regex editing, but that always seemed rather esoteric. Would there be any use or demand for that, do you think?
> remembering the exact syntax of each of the different common regex engines
I have the same problem. There's no way around those little differences other than to suffer them. It simply isn't feasible to remember all the nuances of the different flavors.
I just know one really well and rely on getting it wrong for the others: guess, test, revise, repeat.
That's one of my pet peeves too. Another pet peeve is that I only touch regex once every few months. When you have a regex set to parse a domain of data you move on to other things. After a while you forget the regex syntax and when you have to parse another set of text or have to debug the older regex, I end up having to relearn a bit of it. But I guess it comes with the territory.
Each one seems to have slightly different syntax, require different characters to be escaped, has different defaults (is global search enabled by default? Multiline? What about case sensitivity?), some don't support certain lookarounds, how does grouping work, and so on.