Complete Beginner's Guide to Regular Expressions for Programming Beginners

This is a complete beginner’s tutorial on regular expressions that’s easy to understand even for programming beginners.

This is a complete guide to regular expressions that I especially want people who aren’t good with computers to learn.

Regular expressions are difficult and deep, but the basics aren’t that hard. It might take time to get used to them, but once you can handle regular expressions, they become a great efficiency tool for both work and personal use.

What Are Regular Expressions

Regular expressions are a feature of programming languages used for text searching and replacement.

In English, they’re called Regular expression, abbreviated as regex or regexp. (The former seems more common.)

Regular expression - Wikipedia

You might not quite understand “text searching and replacement.” Let’s look at some practical examples.

Basic Regular Expression Syntax

Here’s the basic regular expression syntax. Take a quick look through it. A rough understanding is fine. You’ll get used to it as you use it.

Below is the basic regular expression syntax.

| … OR
() … Group
^ … Beginning of line
$ … End of line

Below is a list of character classes.

\s … Whitespace
\S … Non-whitespace
\w … Word character
\W … Non-word character
\d … Digit
\D … Non-digit

Also, there are the following special characters.

\n … Newline
\t … Tab

And the following for range specification.

[abc] … a or b or c
[^abc] … Not a, b or c
[a-z] … One character from a,b,c,…,x,y,z
[A-Z] … One character from A,B,C,…,X,Y,Z
[0-9] … One character from 0,1,2,3,…,8,9
[a-zA-Z0-9] … One character from a,b,c,…,x,y,z A,B,C,…,X,Y,Z 0,1,2,3,…,8,9

Below is a list of quantifiers.

\ … Escape
?= … Positive lookahead
?! … Negative lookahead

Escape means making characters used for regular expression syntax be interpreted as regular characters. For example, \d+ is a regular expression that matches the string 21, but if you want to match the string \d+ itself, you need to escape it like \\d+.

Groups are used when doing regular expression replacement, or when using them in programming languages. They’re also often used when using OR with |.

Positive lookahead and negative lookahead are a bit difficult, so it’s okay to understand them later.

Regular Expression Usage Examples

Let me introduce from simple ones.

I want to search for the strings "regular expression" or "regex"

You've been studying regular expressions and have been saving memos about regular expressions to files. However, you have many memos about regular expressions, some of which are written in English as `regex`. You want to search for all regular expression related strings. Now, it's time for regular expressions.

regular expression|regex

This is the most basic form of regular expression. | means “or”, and is synonymous with the OR search often used in Google search.

I want to search for the strings "regular expression" or "regex" or "regexp"

`regular expression|regex|regexp`

Of course, you can make it three.

I want to search for "regular expression" or "regex", but not "regexp"

Let's say the word regexp already exists, that word has nothing to do with regular expressions, and you've used the word "regexp" many times in your files. And you want to search for regular expression related text `regex` or `regular expression`, but searching with `regular expression|regex` also catches `regexp`. What should you do?

regular expression|regex(?!p)

I want to search for URLs with regular expressions

A particularly common need for regular expressions is URLs and HTML tags. These regular expressions come up immediately with many examples when you search Google, but let me introduce them.

https?:\/\/.*\.png

I want to do an AND search

In Google, you search by separating keywords with spaces, which is called AND search. If you search for `regular expression beginner`, pages containing both "regular expression" and "beginner" will appear. The same thing is possible with regular expressions.

^(?=.*regular expression)(?=.*beginner)

I want to do a minus search

Google also has a minus search feature that excludes a keyword like `-advanced`.

^(?=.*regular expression)(?!.*advanced)

I want to do an OR search

Let's put together the regular expressions we've learned so far. You want to search for memos about regular expressions, regardless of English or Japanese, excluding "advanced."

^((?=.*regular expression)(?!.*advanced)|(?=.*regex)(?!.*high))

This was just a small part, but I’ve briefly introduced some useful regular expressions. Please try them out yourself.

Try Out Regular Expressions

The recommended way to actually try out regular expressions is to practice them in a text editor. Many programming text editors like atom have regular expression features built in. I think the easiest way to understand is to actually do regular expression searches and replacements on your preferred document data in these text editors.

That said, there are also quick test methods. There are also several websites for testing regular expressions, and many come up when you search. The following site is one of them. Please make use of it.

RegExr: Learn, Build, & Test RegEx

Regular Expressions Aren’t Just One Kind

Even when we say regular expressions, there are actually several types, and there are versions too. If you continue using regular expressions without being clear about these, eventually you might find yourself thinking “Huh? It should be correct but the regular expression isn’t working…” Let’s understand this roughly.

First, you might think that regular expressions are just regular expressions, but there are several types.

First, the representative one is PCRE (Perl Compatible Regular Expressions).

Perl Compatible Regular Expressions - Wikipedia

This is adopted in programming languages like PHP and Perl.

Next is POSIX regular expressions, which is more minor than PCRE and has slightly different syntax. Database languages like MySQL fall into this category.

Like search, also called fuzzy search, is also a type of regular expression.

Also, if you’re familiar with programming, you’ve probably used command line wildcard searches. You use them like ls *.txt. This is also a variant of regular expressions. Strictly speaking, these features are named Globbing or Bash Extended Globbing.

Not well known, but with Bash Extended Globbing, you can also write syntax closer to regular regular expressions like the following. It’s an interesting example with completely different syntax from regular regular expressions.

*.+(flac|wav|wave)

Remember that regular expressions in one environment or language may not work the same as regular expressions in others. That said, most regular expressions are compatible, so it’s a matter of upward compatibility (Ruby, PHP, Perl, etc.) or downward compatibility (JavaScript, MySQL, etc.). Even if you don’t understand programming, it’s good to look at what programming language the software that can use regular expressions is made in.

Let’s Try Regular Expression Search in Chrome Browser

Now, I think many people use Chrome for browsing, and JavaScript, a programming language, runs in the background in web browsers like Chrome. Chrome extensions are also written in JavaScript, and there are several extensions that provide regular expression functionality.

Chrome extension for regular expression search

As mentioned earlier, JavaScript is not PCRE but something more basic, so its functionality is slightly limited. (Specifically, features like negative lookbehind cannot be used.) That said, text editors like atom are also made from JavaScript, so they use the same regular expressions.

Software That Incorporates Regular Expressions

Let me introduce some useful software that incorporates regular expression search and replacement.

everything

everything is a file search software that can be used on Windows. It’s a type that searches file names, not text data.

I’ve introduced specific regular expressions for everything in another article.

Super-efficiency Windows 10 file search with everything

atom

atom is a popular text editor in recent years.

Atom

I think the most common opportunity to use regular expression search is in a text editor. Not only regular expression search but also regular expression replacement can be used.

Example of regular expression replacement to convert image URL to Markdown

(https:.*yuis\.xsrv\.jp.*(\.png|\.jpg))
![$1]($1)

ag

ag is a command line utility that regular expression searches text files and command output.

I’ve introduced specific regular expressions for ag in another article.

ag is the best for full-text regular expression search in Bash

Sites to Learn Deeper Regular Expressions

The basic regular expressions are as described above, but regular expressions go much deeper.

Although it’s in English, this site explains regular expressions in great depth.

Regex Cheat Sheet

Also, regex repl - Google Search

Regular Expressions Are Deep

I’ve been touching regular expressions since I started programming several years ago, and I’ve finally come to understand the basics. Regular expressions are quite deep and require study as a category, but I think they’re well worth it.

At the very least, if the concept of regular expressions disappeared, many people including me would become depressed. Lol That’s how powerful a tool regular expressions are. Please start learning from the simple parts.