{"id":646,"date":"2011-10-05T21:47:00","date_gmt":"2011-10-05T21:47:00","guid":{"rendered":"http:\/\/www.javacodegeeks.com\/2012\/10\/scala-tutorial-regular-expressions-matching-and-substitutions-with-the-scala-util-matching-api.html"},"modified":"2012-10-21T20:32:10","modified_gmt":"2012-10-21T20:32:10","slug":"scala-tutorial-regular-expressions_05","status":"publish","type":"post","link":"https:\/\/www.javacodegeeks.com\/2011\/10\/scala-tutorial-regular-expressions_05.html","title":{"rendered":"Scala Tutorial – regular expressions, matching and substitutions with the scala.util.matching API"},"content":{"rendered":"
\n

Preface<\/h2>\n

This is part 6 of tutorials for first-time programmers getting into Scala. Other posts are on this blog, and you can get links to those and other resources on the links page of the Computational Linguistics course<\/a> I\u2019m creating these for. Additionally you can find this and other tutorial series on the JCG Java Tutorials<\/a> page.<\/p>\n

This post is the second of two about regular expressions (regexes), which are essential for a wide range of programming tasks, and for computational linguistics tasks in particular. If you haven\u2019t read it already, you might want to start with the first post about regexes<\/a>. For what its worth, this post might actually be of some use to programmers who already are reasonably familiar with Scala but who haven\u2019t used regular expressions much yet: it might saving some poking around to figure out how to do things you already know how to do quite well in other languages.<\/p>\n

The use of regular expressions for capturing values for variable assignment and cases in match expressions is a very clean, well-thought out and highly useful trait of support for regular expressions in the Scala language. However, their use for more complex string matching and substitution is, frankly, much less straightforward than it is in languages with built-in support for regular expressions, such as Perl (which\u2014speaking as one who has coded a lot in Perl\u2014you do *not* want to use for general programming). Scala is fully capabable in that you can use regular expressions fully, but you\u2019ll need to use it via the Regex API. In other words, you need to use a number of commands, not all of which as as straightforward as they could be. (This is not a rant, though I do obviously wish regular expressions were supported more naturally in Scala.)<\/p>\n

Though I\u2019ll refer to what I\u2019m doing below as using the Regex API, I\u2019ll note first that this makes it sound like a bigger deal than it is. It just means you are directly using classes and objects from the scala.util.matching<\/strong> package<\/a> rather than using some of the special syntax and integration with Scala pattern matching we saw in the previous post.<\/p>\n

More extensive matching<\/h2>\n

First off, let\u2019s do what we did with pattern matching in the previous post, but now using the Regex<\/strong> class and the methods available to it to achieve the same ends. We can then start working with finding multiple matches and performing substitutions.<\/p>\n

To recap, recall the name regular expression and how we can use it to initialize a group of variables based on matching a given string.<\/p>\n

scala> val Name = \"\"\"(Mr|Mrs|Ms)\\. ([A-Z][a-z]+) ([A-Z][a-z]+)\"\"\".r\r\nName: scala.util.matching.Regex = (Mr|Mrs|Ms)\\. ([A-Z][a-z]+) ([A-Z][a-z]+)\r\n \r\nscala> val smith = \"Mr. John Smith\"\r\nsmith: java.lang.String = Mr. John Smith\r\n \r\nscala> val Name(title, first, last) = smith\r\ntitle: String = Mr\r\nfirst: String = John\r\nlast: String = Smith\r\n<\/pre>\n
Instead of doing it this way, let\u2019s instead use the API methods. We start by using the regex to find the matches, if any. The method findAllIn<\/strong> of Regex does this for us.<\/p>\n
scala> val matchesFound = Name.findAllIn(smith)\r\nmatchesFound: scala.util.matching.Regex.MatchIterator = non-empty iterator\r\n<\/pre>\n
The result is an iterator<\/em>, which is an object that is like a list in that you can iterate over its elements with for<\/strong> expressions and foreach<\/strong>, use map<\/strong> to transform its values, and more.<\/p>\n
scala> matchesFound.foreach(println)\r\nMr. John Smith\r\n<\/pre>\n
However, unlike Lists, you can only do this a single time. As the following shows, after you iterate through it once, its elements are used up.<\/p>\n
scala> val matchesFound = Name.findAllIn(smith)\r\nmatchesFound: scala.util.matching.Regex.MatchIterator = non-empty iterator\r\n \r\nscala> matchesFound.foreach(println)\r\nMr. John Smith\r\n \r\nscala> matchesFound.foreach(println)\r\n<\/pre>\n
Another difference is that you cannot index into its elements directly.<\/p>\n
scala> val matchesFound = Name.findAllIn(smith)\r\nmatchesFound: scala.util.matching.Regex.MatchIterator = non-empty iterator\r\n \r\nscala> matchesFound(0)\r\n<console>:11: error: scala.util.matching.Regex.MatchIterator does not take parameters\r\nmatchesFound(0)\r\n^\r\n<\/pre>\n
If you wish to do that, you need to just call toList<\/strong> on the MatchIterator<\/strong>.<\/p>\n
scala> val matchList = Name.findAllIn(smith).toList\r\nmatchList: List[String] = List(Mr. John Smith)\r\n \r\nscala> matchList.foreach(println)\r\nMr. John Smith\r\n \r\nscala> matchList.foreach(println)\r\nMr. John Smith\r\n<\/pre>\n
I\u2019ll primarily work with the match results as a List for the remainder of this tutorial. However, note that when you are programming, you should consider whether you really need to do this\u2014usually, the iterator will be sufficient and it has the advantage of being a more efficient.<\/p>\n
Note above that what we have is a List[String]<\/strong>. That means we can see which portions of a string matched, which could include multiple matches.<\/p>\n
scala> val sentence = \"Mr. John Smith said hello to Ms. Jane Hill and then to Mr. Bill Brown.\"\r\nsentence: java.lang.String = Mr. John Smith said hello to Ms. Jane Hill and then to Mr. Bill Brown.\r\n \r\nscala> val matchList = Name.findAllIn(sentence).toList\r\nmatchList: List[String] = List(Mr. John Smith, Ms. Jane Hill, Mr. Bill Brown)\r\n<\/pre>\n
This will be useful in many contexts, but it won\u2019t allow us to access the match groups that were defined in the Regex. For that, we need to use the matchData<\/strong> method, which converts the MatchIterator (which offers Strings as its elements) into an Iterator[Match]<\/strong> (which offers Match objects as its elements).
 
<\/div> <\/div><\/p>\n
scala> val matchList = Name.findAllIn(smith).matchDatamatchList: java.lang.Object with Iterator[scala.util.matching.Regex.Match] = non-empty iterator\r\n<\/pre>\n
Let\u2019s convert that to a List and then grab the first element.<\/p>\n
scala> val matchList = Name.findAllIn(smith).matchData.toList\r\nmatchList: List[scala.util.matching.Regex.Match] = List(Mr. John Smith)\r\n \r\nscala> val firstMatch = matchList(0)\r\nfirstMatch: scala.util.matching.Regex.Match = Mr. John Smith\r\n<\/pre>\n
This Match object contains captured groups that we can access with the group method. The first index, 0, returns the entire match, and the rest access the captured groups.<\/p>\n
scala> firstMatch.group(0)\r\nres8: String = Mr. John Smith\r\n \r\nscala> val title = firstMatch.group(1)\r\ntitle: String = Mr\r\n \r\nscala> val first = firstMatch.group(2)\r\nfirst: String = John\r\n \r\nscala> val last = firstMatch.group(3)\r\nlast: String = Smith\r\n<\/pre>\n
We can get a bit closer to the original pattern matched variable assignment by packaging them up as a tuple.<\/p>\n
scala> val (title, first, last) = (firstMatch.group(1), firstMatch.group(2), firstMatch.group(3))\r\ntitle: String = Mr\r\nfirst: String = John\r\nlast: String = Smith\r\n<\/pre>\n
Update<\/em>: There is a more concise way to do this using the range 1 to 3 and map firstMatch.group over that range. This creates a Seq(uence), which we can pattern match on. (Thanks to @missingfaktor.)<\/p>\n
val Seq(title, first, last) = 1 to 3 map firstMatch.group\r\n<\/pre>\n
This should demonstrate why Scala\u2019s support for Regexes in patterning match is very nice for this. What you gain with the API is the ability to match multiple instances of a pattern in a string and then to perform computations with the Match results on the fly. For example, let\u2019s return to the sentence with multiple names in it and use the Name regex to say hello to every name found in it.<\/p>\n
scala> Name.findAllIn(sentence).matchData.foreach(m => println(\"Hello, \" + m.group(0)))\r\nHello, Mr. John Smith\r\nHello, Ms. Jane Hill\r\nHello, Mr. Bill Brown\r\n<\/pre>\n
Of course, you can choose to print only subparts of the names, such as the title and the last name.<\/p>\n
scala> Name.findAllIn(sentence).matchData.foreach(m => println(\"Hello, \" + m.group(1) + \". \" + m.group(3)))\r\nHello, Mr. Smith\r\nHello, Ms. Hill\r\nHello, Mr. Brown\r\n<\/pre>\n
Or you can filter the results, e.g. to only the Mr\u2019s, and then print only the first names.<\/p>\n
scala> Name.findAllIn(sentence).matchData.filter(m=>m.group(1) == \"Mr\").foreach(m => println(\"Hello, \" + m.group(2)))\r\nHello, John\r\nHello, Bill\r\n<\/pre>\n
Notice that in the above lines, I didn\u2019t convert the MatchIterator to a List since I was happy to just go through the list once and do some actions.<\/p>\n
         Performing substitutions<\/h2>\n
The other thing you gain is the ability to use regular expressions for substituting once class of expressions with another. For example, let\u2019s say that (for some odd reason) you would like to reverse everyone\u2019s name so that \u201cMr. John Smith<\/em>\u201d becomes \u201cMr. Smith John<\/em>\u201c. This is accomplished by using the Regex method replaceAllIn<\/strong>, which takes two arguments: the first is the original string and the second is a function that takes a Match object and returns a String.<\/p>\n
scala> val swapped = Name.replaceAllIn(sentence, m => m.group(1) + \". \" + m.group(3) + \" \" + m.group(2))\r\nswapped: String = Mr. Smith John said hello to Ms. Hill Jane and then to Mr. Brown Bill.\r\n<\/pre>\n
The variable m<\/em> above is referring to each of the Match objects identified, in turn. That means we can access the groups as we did before. The thing that might feel strange at first is that the anonymous function m => m.group(1) + \u201c. \u201d + m.group(3) + \u201d \u201d + m.group(2)<\/strong> is an argument. It\u2019s not very different from the following, where we first create a named function and then pass it as an argument.<\/p>\n
scala> def swapFirstLast = (m: scala.util.matching.Regex.Match) => m.group(1) + \". \" + m.group(3) + \" \" + m.group(2)\r\nswapFirstLast: (util.matching.Regex.Match) => java.lang.String\r\n \r\nscala> val swapped = Name.replaceAllIn(sentence, swapFirstLast)swapped: String = Mr. Smith John said hello to Ms. Hill Jane and then to Mr. Brown Bill.\r\n<\/pre>\n
Note that now that we\u2019ve defined it, we can use that same function to map the Matches returned by findAllIn<\/strong> to their swapped versions.<\/p>\n
scala> val swappedNames = Name.findAllIn(sentence).matchData.map(swapFirstLast).toList\r\nswappedNames: List[java.lang.String] = List(Mr. Smith John, Ms. Hill Jane, Mr. Brown Bill)\r\n<\/pre>\n
The difference is that using findAllIn<\/strong> gives us the Match results themselves, whereas replaceAllIn<\/strong> replaces them in the String in situ. Whether you need to do one or the other depends on your programming needs.<\/p>\n
         Determining whether an entire string matches using the Regex API<\/h2>\n
If you just want to know whether an entire given string matches a Regex, Scala unfortunately has a somewhat roundabout way for you to do this. First, here is the syntax, testing whether Name matches on the variables smith<\/em> and sentence<\/em>.<\/p>\n
scala> Name.pattern.matcher(smith).matches\r\nres21: Boolean = true\r\n \r\nscala> Name.pattern.matcher(sentence).matches\r\nres22: Boolean = false\r\n<\/pre>\n
So, sentence<\/em> doesn\u2019t match (despite having three names in it) because the entire string is not a single match to Name.<\/p>\n
What is going on here is that we are actually using classes defined in Java for working with regular expressions. First, we get the java.util.regex.Pattern<\/strong> object associated with our scala.util.matching.Regex<\/strong> object.<\/p>\n
scala> Name.pattern\r\nres16: java.util.regex.Pattern = (Mr|Mrs|Ms)\\. ([A-Z][a-z]+) ([A-Z][a-z]+)\r\n<\/pre>\n
Then we use that Pattern to get a java.util.regex.Matcher<\/strong> for the string.<\/p>\n
scala> Name.pattern.matcher(smith)\r\nres17: java.util.regex.Matcher = java.util.regex.Matcher[pattern=(Mr|Mrs|Ms)\\. ([A-Z][a-z]+) ([A-Z][a-z]+) region=0,14 lastmatch=]\r\n<\/pre>\n
The Matcher class has a matches<\/strong> method that tells us whether there was a match or not for that string.<\/p>\n
scala> Name.pattern.matcher(smith).matches\r\nres18: Boolean = true\r\n<\/pre>\n
So, long-winded, but you can do it.<\/p>\n
Note<\/em>: there is another way to do this using Scala\u2019s standard pattern matching paradigm discussed in the previous post on regexes<\/a>.<\/p>\n
scala> smith match { case Name(_,_,_) => true; case _ => false }\r\nres23: Boolean = true\r\n \r\nscala> sentence match { case Name(_,_,_) => true; case _ => false }\r\nres24: Boolean = false\r\n<\/pre>\n
However, this requires the extra work of specifying the capture groups, which are being thrown away anyway.<\/p>\n
         Simple substitutions with a second regular expression<\/h2>\n
There is another replaceAllIn<\/strong> method that takes a String defining a (fairly) standard regular expresion substitution as its second argument rather than a function from Matches to Strings. This argument defines a regular expression similar to that used in standard s\/\/\/<\/em> substitutions from the Perl programming language,e.g. the following, which turns strings like \u201cxyzaaaabbb123<\/em>\u201d int \u201cxyzbbbaaaa123<\/em>\u201c.<\/p>\n
s\/(a+)(b+)\/\\2\\1\/\r\n<\/pre>\n
Unlike Perl (which is the same as the syntax discussed in Jurafsky and Martin\u2019s book), Scala uses $1<\/strong>, $2<\/strong>, etc. As an example, consider the first-last name swap we did before. Here it is repeated:<\/p>\n
scala> val swapped = Name.replaceAllIn(sentence, m => m.group(1) + \". \" + m.group(3) + \" \" + m.group(2))\r\nswapped: String = Mr. Smith John said hello to Ms. Hill Jane and then to Mr. Brown Bill.\r\n<\/pre>\n
You can get the exact same effect somewhat more easily by constructing the replacement string with $n<\/strong> variables that refer to the groups.<\/p>\n
scala> val swapped2 = Name.replaceAllIn(sentence, \"$1. $3 $2\")\r\nswapped2: String = Mr. Smith John said hello to Ms. Hill Jane and then to Mr. Brown Bill.\r\n<\/pre>\n
This is far more concise and readable than the m.group()<\/strong> style above, so it is preferable for cases like this. However, sometimes you\u2019ll want to do some more interesting processing of the values in each group, such as changing the titles to another language and outputing only the first initial of the first name: e.g. \u201cMr. John Smith<\/em>\u201d would become \u201cSr. J. Smith<\/em>\u201d and \u201cMrs. Jane Hill<\/em>\u201d would become \u201cSra. J. Hill\u201d. It isn\u2019t clear to me how one could do this with the $n<\/strong> substitutions (if some reader is aware, please let me know). To do it with the Match => String<\/strong> function, it is straightforward. First, let\u2019s define a method that maps the titles from English to Spanish.<\/p>\n
def engTitle2Esp (title: String) = title match {\r\n  case \"Mr\" => \"Sr\"\r\n  case \"Mrs\" => \"Sra\"\r\n  case \"Ms\" => \"Srta\"\r\n}\r\n<\/pre>\n
Then we pass m.group(1)<\/strong> through that function by using engTitle2Esp(m.group(1))<\/strong>, and get just the first character of group 2 by indexing into it as m.group(2)(0)<\/strong>.<\/p>\n
scala> val spanishized = Name.replaceAllIn(sentence, m => engTitle2Esp(m.group(1)) + \". \" + m.group(2)(0) + \". \" + m.group(3))\r\nspanishized: String = Sr. J. Smith said hello to Srta. J. Hill and then to Sr. B. Brown.\r\n<\/pre>\n
This gives you considerable control over how to process the replacements.<\/p>\n
Reference: <\/i><\/strong>First steps in Scala for beginning programmers, Part 6<\/a> from our JCG partner<\/a> Jason Baldridge at the Bcomposes<\/a> blog.<\/p>\n
Related Articles :<\/i><\/strong><\/p>\n