{"id":44575,"date":"2015-09-27T22:07:45","date_gmt":"2015-09-27T19:07:45","guid":{"rendered":"http:\/\/www.javacodegeeks.com\/?p=44575"},"modified":"2023-12-06T15:41:20","modified_gmt":"2023-12-06T13:41:20","slug":"lucene-query-search-syntax-examples","status":"publish","type":"post","link":"https:\/\/www.javacodegeeks.com\/2015\/09\/lucene-query-search-syntax-examples.html","title":{"rendered":"Lucene Query (Search) Syntax Examples"},"content":{"rendered":"

This article is part of our Academy Course titled Apache Lucene Fundamentals<\/a>.<\/p>\n

In this course, you will get an introduction to Lucene. You will see why a library like this is important and then learn how searching works in Lucene. Moreover, you will learn how to integrate Lucene Search into your own applications in order to provide robust searching capabilities. Check it out here<\/a>!<\/em><\/p>\n

\n

Table Of Contents<\/strong><\/p>\n

\n
1. Introduction<\/a><\/dt>\n
2. The Query Class<\/a><\/dt>\n
\n
\n
2.1 TermQuery<\/a><\/dt>\n
2.2 PhraseQuery<\/a><\/dt>\n
2.3. BooleanQuery<\/a><\/dt>\n
2.4. WildcardQuery<\/a><\/dt>\n
2.5. RegexpQuery<\/a><\/dt>\n
2.6. TermRangeQuery<\/a><\/dt>\n
2.7. NumberRangeQuery<\/a><\/dt>\n
2.8. FuzzyQuery<\/a><\/dt>\n
\n\t<\/dd>\n<\/dl>\n<\/div>\n

1. Introduction<\/h2>\n

In this lesson of our course we are going to investigate the basic querying mechanisms offerd by Lucene. As you might remember from the introductory lesson, Lucene does not send raw text to be searched to the index. It uses Query<\/code> Objects for that. In this lesson we are going to see all the crucial components that line up, to convert human – written search phrases to representative structures like Queries<\/code>.<\/p>\n

2. The Query Class<\/h2>\n

The Query<\/code> class is a public abstract class that represents a query to the index. In this section we are going to see the most important Query sub – classes that you can use to perform highly tailored queries.<\/p>\n

2.1 TermQuery<\/h3>\n

This is the most simple and straightforward query you can perform against a Lucene index. You simply search for Documents<\/code> that cointain a single word in a specific Field<\/code>.<\/p>\n

The basic TermQuery<\/code> constructor is defined as follows : public TermQuery(Term t)<\/code>. As you remember from the fist lesson a Term<\/code> consists of a two parts:<\/p>\n

    \n
  1. The name of the Field<\/code> in which this term resides.<\/li>\n
  2. The actual value of the Term, which, in the great majority of cases, is a single word, obtained from the analysis of some plain text.<\/li>\n<\/ol>\n

    So, if you want to create a TermQuery to find all Documents<\/code> that contain the word \"good\"<\/code> in their \"content\"<\/code> Field<\/code>, here’s how you can do it<\/p>\n

    \nTermQuery termQuery = new TermQuery(new Term(\"content\",\"good\"));\n<\/pre>\n
    We can use that to search for the word “static” in our previously created index:<\/p>\n
    \nString q = \"static\"\n\nDirectory directory = FSDirectory.open(indexDir);\n\nIndexReader  indexReader  = DirectoryReader.open(directory);\n\nIndexSearcher searcher = new IndexSearcher(indexReader);\n\nAnalyzer analyzer = new StandardAnalyzer(Version.LUCENE_46);\n\nTermQuery termQuery = new TermQuery(new Term(\"content\",q));\n\nTopDocs topDocs =searcher.search(termQuery, maxHits);\n\nScoreDoc[] hits = topDocs.scoreDocs;\n\nfor (ScoreDoc hit : hits) {\n        int docId = hit.doc;\n        Document d = searcher.doc(docId);\n        System.out.println(d.get(\"fileName\") + \" Score :\" + hit.score);\n}\n\nSystem.out.println(\"Found \" + hits.length);\n<\/pre>\n
    The output of this would be:<\/p>\n
    \nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\Product.java Score :0.29545835\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\SimpleSearcher.java Score :0.27245367\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\PropertyObject.java Score :0.24368995\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\SimpleIndexer.java Score :0.14772917\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\TestSerlvet.java Score :0.14621398\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\ShoppingCartServlet.java Score :0.13785185\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\MyServlet.java Score :0.12184498\nFound 7\n<\/pre>\n
    As you can see, seven of my source files contained the \"static\"<\/code> keyword. That is there is to it. Naturally, if you try to add another word in the query string, the search will return 0 results. For example if you set your query string to:<\/p>\n
    \nString q = \"private static\"\n<\/pre>\n
    The output would be :<\/p>\n
    \nFound 0\n<\/pre>\n
    Now, I know that \"private static\"<\/code> is present inside many of my source files. But as you might remember, we used the StandarAnalyzer<\/code> to process the plain text retrieved from our files, in the indexing process. StandardAnalyzer<\/code> splits the text into individual words, thus every Term<\/code> contains one single word. You can choose not to tokenize an indexed Field<\/code>. But I would suggest that you do that in Fields<\/code> that contain meta – information about our Document, e.g. the title or the author, and not in the Fields that hold its content. For example if you choose not to tokenize and indexed Field with name 'author'<\/code> and value 'James Wilslow'<\/code>, the Field<\/code> 'author'<\/code> will contain only one Term<\/code> with value 'James Wilslow'<\/code> as a whole. If you did tokenize the Field<\/code>, it would contain two Terms<\/code>, one with value 'James'<\/code> and the other one with value 'Wilslow'<\/code> .<\/p>\n
    2.2 PhraseQuery<\/h3>\n
    With PhraseQuery<\/code> you can search for Documents<\/code> that contain a particular sequence of words, aka phrases.<\/p>\n
    You can create a PhraseQuery<\/code> like this:<\/p>\n
    \nPhraseQuery phraseQuery = new PhraseQuery();\n<\/pre>\n
    And then you can add Terms<\/code> to it. For example if you want to search for Documents<\/code> that contain the phrase “private static” in their “content” Field, you could do it like so:<\/p>\n
    \nPhraseQuery phraseQuery = new PhraseQuery();\n\nphraseQuery.add(new Term(\"content\",\"private\"));\nphraseQuery.add(new Term(\"content\",\"static\"));\n\nTopDocs topDocs =searcher.search(phraseQuery, maxHits);\n\nScoreDoc[] hits = topDocs.scoreDocs;\n\nfor (ScoreDoc hit : hits) {\n      int docId = hit.doc;\n      Document d = searcher.doc(docId);\n      System.out.println(d.get(\"fileName\") + \" Score :\" + hit.score);\n}\n\nSystem.out.println(\"Found \" + hits.length);\n<\/pre>\n
    The output would be :
     
    <\/div> <\/div><\/p>\n
    \nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\Product.java Score :0.54864377\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\PropertyObject.java Score :0.45251375\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\SimpleSearcher.java Score :0.45251375\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\TestSerlvet.java Score :0.27150828\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\ShoppingCartServlet.java Score :0.25598043\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\MyServlet.java Score :0.22625688\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\SimpleIndexer.java Score :0.22398287\nFound 7\n<\/pre>\n
    A Document<\/code> makes it into the results, only if the Field<\/code> contains both the words \"private\"<\/code> and \"static\"<\/code>, consecutively and in that exact order.<\/p>\n
    So if you change the above code in something like this:<\/p>\n
    \nphraseQuery.add(new Term(\"content\",\"private\"));\nphraseQuery.add(new Term(\"content\",\"final\"));\n<\/pre>\n
    You will get :<\/p>\n
    \nFound 0\n<\/pre>\n
    That’s because although my source files contains both of the words, the are not consecutive. To alter that behavior a little bit you can add a slop<\/code> to the PhraseQuery<\/code>. When you add a slop of 1, you allow at most one word to intervene between the words on your phrase. When you add slop 2, you allow at most 2 words between your words on the phrase.<\/p>\n
    Interestingly : “The slop is in fact an edit-distance, where the units correspond to moves of terms in the query phrase out of position. For example, to switch the order of two words requires two moves (the first move places the words atop one another), so to permit re-orderings of phrases, the slop must be at least two.”<\/em><\/p>\n
    So if we do:<\/p>\n
    \nPhraseQuery phraseQuery = new PhraseQuery();\n\nphraseQuery.add(new Term(\"content\",\"private\"));\nphraseQuery.add(new Term(\"content\",\"final\"));\n\nphraseQuery.setSlop(2);\n<\/pre>\n
    The output of our search will give:<\/p>\n
    \nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\Product.java Score :0.38794976\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\PropertyObject.java Score :0.31997555\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\SimpleSearcher.java Score :0.31997555\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\TestSerlvet.java Score :0.19198532\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\ShoppingCartServlet.java Score :0.18100551\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\MyServlet.java Score :0.15998778\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\SimpleIndexer.java Score :0.15837982\n<\/pre>\n
    It is important to mention that documents that contain phrases closer to the exact phrase of the query, will get higher scores.<\/p>\n
    2.3 BooleanQuery<\/h3>\n
    BooleanQuery<\/code> is a more expressive and powerful tool, as you can combine multiple Queries together with Boolean clauses. A BoleanQuery can be populated with BooleanClauses. A BooleanClause consists of a Query<\/code>, and the role that that Query<\/code> should have on the boolean search.<\/p>\n
    To be more specific, a boolean clause can play the following roles in a query:<\/p>\n
      \n
    1. MUST<\/code> : This is pretty self explenatory<\/code>. A Document<\/code> makes it to the list of the results, if and only if it contains that clause.<\/li>\n
    2. MUST NOT<\/code> : It’s the exact opposite case. It is obligatory, for a Document to make it to the result list, not to contain that clause.<\/li>\n
    3. SHOULD<\/code> : This is for clauses that can occur in a Document<\/code>, but it is not necessary for them to include it in order to make it to the results.<\/li>\n<\/ol>\n
      If you have a boolean query only with SHOULD clauses the results matches at least one of the clauses. This seems like the classic OR boolean operator but it is not so straightforward to use it properly.<\/p>\n
      Now let’s see some examples. Let’s find the source files that contain the word “string” but don’t contain the word “int”.<\/p>\n
      \nTermQuery termQuery = new TermQuery(new Term(\"content\",\"string\"));\nTermQuery termQuery2 = new TermQuery(new Term(\"content\",\"int\"));\n\nBooleanClause booleanClause1 = new BooleanClause(termQuery, BooleanClause.Occur.MUST);\nBooleanClause booleanClause2 = new BooleanClause(termQuery2, BooleanClause.Occur.MUST_NOT);\n\nBooleanQuery booleanQuery = new BooleanQuery();\nbooleanQuery.add(booleanClause1);\nbooleanQuery.add(booleanClause2);\n\nTopDocs topDocs =searcher.search(booleanQuery, maxHits);\n<\/pre>\n
      Here is the result:<\/p>\n
      \nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\SimpleEJB.java Score :0.45057273\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\PropertyObject.java Score :0.39020744\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\ShoppingCartServlet.java Score :0.20150226\nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\TestSerlvet.java Score :0.13517183\nFound 4\n<\/pre>\n
      Now let’s try to find all the Documents that contain the word “nikos” and the phrase “httpservletresponse response”. In the following snippet you can see how you can avoid creating BooleanClause<\/code> instances, making your more compact.<\/p>\n
      \nTermQuery termQuery = new TermQuery(new Term(\"content\",\"nikos\"));\n\nPhraseQuery phraseQuery = new PhraseQuery();\nphraseQuery.add(new Term(\"content\",\"httpservletresponse\"));\nphraseQuery.add(new Term(\"content\",\"response\"));\n\nBooleanQuery booleanQuery = new BooleanQuery();\n\nbooleanQuery.add(phraseQuery,BooleanClause.Occur.MUST);\nbooleanQuery.add(termQuery,BooleanClause.Occur.MUST);\n\nTopDocs topDocs =searcher.search(booleanQuery, maxHits);\n<\/pre>\n
      This is the result:<\/p>\n
      \nC:\\\\Users\\\\nikos\\\\Desktop\\\\LuceneFolders\\\\LuceneHelloWorld\\\\SourceFiles\\\\ShoppingCartServlet.java Score :0.3148332\nFound 1\n<\/pre>\n
      Let’s find all the Documents that contain the word “int” or the word “nikos”.As you might image you must use the SHOULD specification somehow:<\/p>\n
      \nTermQuery termQuery = new TermQuery(new Term(\"content\",\"int\"));\nTermQuery termQuery2 = new TermQuery(new Term(\"content\",\"nikos\"));\n\nBooleanQuery booleanQuery = new BooleanQuery();\n\nbooleanQuery.add(termQuery,BooleanClause.Occur.SHOULD);\nbooleanQuery.add(termQuery2,BooleanClause.Occur.SHOULD);\n\nTopDocs topDocs =searcher.search(booleanQuery, maxHits);\n<\/pre>\n
      This was not too hard but it is a bit tricky to create more complicated disjunctive queries. It is not always straightforward how you can use SHOULD correctly.<\/p>\n
      For example let’s try to find all the Documents that contain the word “nikos” and the phrase “httpservletresponse response” or contain the word “int”. One could write something like this:<\/p>\n
      \nTermQuery termQuery = new TermQuery(new Term(\"content\",\"nikos\"));\n\nPhraseQuery phraseQuery = new PhraseQuery();\nphraseQuery.add(new Term(\"content\",\"httpservletresponse\"));\nphraseQuery.add(new Term(\"content\",\"response\"));\n\nBooleanQuery booleanQuery = new BooleanQuery();\n\nbooleanQuery.add(phraseQuery,BooleanClause.Occur.MUST);\nbooleanQuery.add(termQuery,BooleanClause.Occur.MUST);\nbooleanQuery.add(new TermQuery(new Term(\"content\",\"int\")),BooleanClause.Occur.SHOULD);\n\nTopDocs topDocs =searcher.search(booleanQuery, maxHits);\n<\/pre>\n
      But the query would fail to provide the results you want. Remember that the results of this query, as we’ve constructed in, MUST contain the word \"nikos\"<\/code> and MUST contain the phrase \"httpservletresponse response\"<\/code> in the same time. But this is not what you want. You want the documents that contain the word nikos and the phrase \"httpservletresponse response\"<\/code>, but you also want documents that contain the word \"int\"<\/code> independently, no matter if they contain the other clauses. To be fair, the above boolean query is a bit wrong. Because in straight boolean syntactic you would never write something like: A AND B OR C. You should write (A AND B) OR C. Or A AND (B OR C). See the difference?<\/p>\n
      So you should write the query you want like : ( “nikos” AND “httpservletresponse response” ) OR “int”.<\/p>\n
      You can do that combining BooleanQueries<\/code> together. Using the above strict syntax it is not very hard to imagine how this would go:<\/p>\n
      \nTermQuery termQuery = new TermQuery(new Term(\"content\",\"nikos\"));\n\nPhraseQuery phraseQuery = new PhraseQuery();\nphraseQuery.add(new Term(\"content\",\"httpservletresponse\"));\nphraseQuery.add(new Term(\"content\",\"response\"));\n\n\/\/ (A AND B)\nBooleanQuery conjunctiveQuery = new BooleanQuery();\nconjunctiveQuery.add(termQuery,BooleanClause.Occur.MUST);\nconjunctiveQuery.add(phraseQuery,BooleanClause.Occur.MUST);\n\nBooleanQuery disjunctiveQuery = new BooleanQuery();\n\n\/\/ (A AND B) OR C\ndisjunctiveQuery.add(conjunctiveQuery,BooleanClause.Occur.SHOULD);\ndisjunctiveQuery.add(new TermQuery(new Term(\"content\",\"int\")),BooleanClause.Occur.SHOULD);\n\nTopDocs topDocs =searcher.search(disjunctiveQuery, maxHits);\n<\/pre>\n
      This is a quick guide you can follow when constructing boolean queries using BooleanQuery<\/code> class:<\/p>\n