Before we start, please make sure your computer meets the system requirements<\/a>. Also, please download the binary release of Apache Solr 8.5<\/a>. When searching something in Solr, a common way is to specify terms as keywords in a query. The relevance score of each matching document in the search results is then calculated based on the terms’ TF-IDF similarity. The relevance score is used to describe the degree in which a search result satisfies a user searching for information. The higher the relevance score the better a user’s requirement is met. Is there a way for us to generate relevance scores by using our custom calculation in a query so that the search results can satisfy our users’ needs in different contexts? Function queries are introduced for this purpose.<\/p>\n A function query is a special query that can be added to a query and it allows us to specify a function to generate a relevance score at query time for each document in the search results. In addition, the calculated score can then be used to filter out documents, sort results, and append as a field for each document returned.<\/p>\n The following query parsers support function queries:<\/p>\n Function queries use functions. The standard function syntax in Solr consists of a function name, an opening-round bracket, a list of parameters, and a closing round bracket.<\/p>\n In addition to the standard function syntax, there are three simplified function syntax as below:<\/p>\n Note that Solr defines the input parameter types of a function as functions themselves. It means we can pass a function as a parameter of another function.<\/p>\n There are several ways of using function queries in a Solr query. Before we show you some examples, let us prepare the collection and data for our queries.<\/p>\n Before creating a collection to indexing our data, we need a configset for our collection. A configset is a collection of configuration files such as solrconfig.xml, synonyms.txt, the schema, etc.. There are two example configsets ( Note that when running in SolrCloud mode, configsets are fundamentally stored in ZooKeeper and not the file system. Solr\u2019s For this example, we create our own configset We can see the output as below:<\/p>\n If the Now we can use the Configsets API<\/a> to list all configsets on the SolrCloud:<\/p>\n There are two configsets in the response:<\/p>\n Assuming that you have followed the steps in Apache Solr Clustering Example<\/a> to get a SolrCloud up and running on your local machine. Open Solr Admin<\/a> in a browser and create a new collection named You will see the following output once documents have been submitted successfully.<\/p>\n We search for articles whose title contains term Click the As we can see from the output above, there are It means if two articles have the same number of views, then the article with less dislikes is more popular than the other one. Let’s see how we can implement this popularity calculation and use it with a function query.[ulp id=’nVHijykNs8bcCQYH’]<\/p>\n Query parsers such as In this query, we add Now we can see that the relevance scores have been updated. The first article has a score of 201.0 and the second article has a score The query above can be written via the It yields the same output:<\/p>\n Function queries can be used in a sort expression. For example:<\/p>\n In this query, instead of using our popularity function for the relevance score, we just add it in the sort expression to sort the results by the popularity in descending order. The output would be:<\/p>\n We can see that the relevance scores remain the same but the article with higher popularity value is put in front.<\/p>\n Another useful scenario is to add the calculation results as fields of documents in the search results. For instance:<\/p>\n In this query, we add a pseudo-field There is a list of available Function Queries available here<\/a>. Also, you can implement your own custom functions and use them in the query which is out of the scope of this example.<\/p>\n
In addition, it will save you some time if you can follow the Apache Solr Clustering Example<\/a> to get a SolrCloud up and running on your local machine.<\/p>\n<\/p>\n3. Function Queries Basics<\/h2>\n
3.1 What Is A Function Query<\/h3>\n
3.2 Query Parsers Supporting Function Queries<\/h3>\n
\n
3.3. Function Syntax<\/h3>\n
numdocs()\nord(myIndexedField)\nmax(myfield,myotherfield,0)<\/pre>\n
\n
18, 3.1415, \"Function Query Example\"<\/pre>\n
\n
author, field(author)<\/pre>\n
\n
q={!func}max($f1,$f2)&f1=views&f2=1000<\/pre>\n4. Solr Function Queries Examples<\/h2>\n
4.1 Upload A Configset<\/h3>\n
_default<\/code> and sample_techproducts_configs<\/code>) in the Solr distribution which can be used when creating collections.<\/p>\n_default<\/code> configset is uploaded to ZooKeeper on initialization. So to use our own configset, we need to create a new one and upload it to ZooKeeper.<\/p>\njcg_example_configs<\/code> for our collection simply by making a copy of the _default configset. Download the source code of this example and copy jcg_example_configs.zip to your working directory. For example, we copy jcg_example_configs.zip<\/code> to D:\\<\/code> on Windows 10. Then run the following command in a Command Prompt to upload a configset:curl -X POST --header \"Content-Type:application\/octet-stream\" --data-binary @jcg_example_configs.zip \"http:\/\/localhost:8983\/solr\/admin\/configs?action=UPLOAD&name=jcg_example_configs\"<\/pre>\n
D:\\>curl -X POST --header \"Content-Type:application\/octet-stream\" --data-binary @jcg_example_configs.zip \"http:\/\/localhost:8983\/solr\/admin\/configs?action=UPLOAD&name=jcg_example_configs\"\n{\n \"responseHeader\":{\n \"status\":0,\n \"QTime\":2203}}<\/pre>\njcg_example_config<\/code> configset has already existed, you can delete it with the following command:<\/p>\ncurl -X DELETE http:\/\/localhost:8983\/api\/cluster\/configs\/jcg_example_configs?omitHeader=true<\/pre>\n
curl http:\/\/localhost:8983\/solr\/admin\/configs?action=LIST<\/pre>\n
D:\\>curl http:\/\/localhost:8983\/solr\/admin\/configs?action=LIST\n{\n \"responseHeader\":{\n \"status\":0,\n \"QTime\":1},\n \"configSets\":[\"jcg_example_configs\",\n \"_default\"]}<\/pre>\n4.2 Indexing Data<\/h3>\n
jcgArticles<\/code> with jcg_example_configs<\/code> config set. Select the newly created jcgArticles<\/code> collection and go to Documents<\/code> screen, Copy the content of articles.csv<\/code> file downloaded from this example and paste into Documents<\/code> text box. Select CSV<\/code> from the drop-down list as Document Type<\/code> and click Submit Document<\/code> button.<\/p>\n![\"Solr](\"https:\/\/examples.javacodegeeks.com\/wp-content\/uploads\/2020\/06\/preparing-data-w.jpg\")
Status: success\nResponse:\n{\n \"responseHeader\": {\n \"rf\": 2,\n \"status\": 0,\n \"QTime\": 467\n }\n}<\/pre>\n4.3 Querying Without Using A Function Query<\/h3>\n
SolrCloud<\/code> by using a field query. Also, we add score<\/code> to the fields list of the search results. Later, We will compare the relevance scores returned by this query to the relevance scores of other queries using function queries.<\/p>\n q=title:*SolrCloud*&fl=*,score<\/pre>\n
![\"Solr](\"https:\/\/examples.javacodegeeks.com\/wp-content\/uploads\/2020\/06\/query-without-function-query-w.jpg\")
Execute Query<\/code> button and the output would be:<\/p>\n{\n \"responseHeader\":{\n \"zkConnected\":true,\n \"status\":0,\n \"QTime\":104,\n \"params\":{\n \"q\":\"title:*SolrCloud*\",\n \"fl\":\"*,score\",\n \"wt\":\"json\",\n \"_\":\"1592054831147\"}},\n \"response\":{\"numFound\":2,\"start\":0,\"maxScore\":1.0,\"docs\":[\n {\n \"id\":\"0818231712\",\n \"category\":[\"solr\"],\n \"title\":[\"Apache SolrCloud Example\"],\n \"published\":true,\n \"author\":[\"Kevin Yang\"],\n \"views\":2000,\n \"likes\":1000,\n \"dislikes\":10,\n \"comments\":200,\n \"publish_date\":\"2020-06-05T00:00:00Z\",\n \"_version_\":1669390419809533952,\n \"score\":1.0},\n {\n \"id\":\"0380014300\",\n \"category\":[\"solr\"],\n \"title\":[\"SolrCloud Tutorial\"],\n \"published\":true,\n \"author\":[\"Roger Goodwill\"],\n \"views\":2000,\n \"likes\":1000,\n \"dislikes\":500,\n \"comments\":10,\n \"publish_date\":\"2020-06-05T00:00:00Z\",\n \"_version_\":1669390419821068288,\n \"score\":1.0}]\n }}<\/pre>\n2<\/code> articles found. Both of them with a score of 1.0 and the number of views is the same 2000. How do we know which article is more popular? You may notice that these two articles have a different number of dislikes. So we can define the popularity of an article as below:<\/p>\npopularity = views \/ dislikes<\/pre>\n
4.4 Querying With A Function Query<\/h3>\n
func<\/code> and frange<\/code> expect function arguments. We can use the built-in Solr function div<\/a> to calculate the popularity. For example:<\/p>\nq=title:*SolrCloud* AND _query_:\"{!func}div(views,dislikes)\"&fq={!frange l=1}dislikes&fl=*,score<\/pre>\n{!func}div(views,dislikes)<\/code> function to the query and include the score in the fields list returned. In addition, fq={!frange l=1}dislikes are used to avoid division by zero issues<\/em>. The output would be:<\/p>\n{\n \"responseHeader\":{\n \"zkConnected\":true,\n \"status\":0,\n \"QTime\":97,\n \"params\":{\n \"q\":\"title:*SolrCloud* AND _query_:\\\"{!func}div(views,dislikes)\\\"\",\n \"fl\":\"*,score\",\n \"fq\":\"{!frange l=1}dislikes\",\n \"wt\":\"json\",\n \"_\":\"1592054952916\"}},\n \"response\":{\"numFound\":2,\"start\":0,\"maxScore\":201.0,\"docs\":[\n {\n \"id\":\"0818231712\",\n \"category\":[\"solr\"],\n \"title\":[\"Apache SolrCloud Example\"],\n \"published\":true,\n \"author\":[\"Kevin Yang\"],\n \"views\":2000,\n \"likes\":1000,\n \"dislikes\":10,\n \"comments\":200,\n \"publish_date\":\"2020-06-05T00:00:00Z\",\n \"_version_\":1669390419809533952,\n \"score\":201.0},\n {\n \"id\":\"0380014300\",\n \"category\":[\"solr\"],\n \"title\":[\"SolrCloud Tutorial\"],\n \"published\":true,\n \"author\":[\"Roger Goodwill\"],\n \"views\":2000,\n \"likes\":1000,\n \"dislikes\":500,\n \"comments\":10,\n \"publish_date\":\"2020-06-05T00:00:00Z\",\n \"_version_\":1669390419821068288,\n \"score\":5.0}]\n }}<\/pre>\n5.0<\/code>. Obviously, the first article is more popular than the second one based on our popularity definition.<\/p>\n_val_<\/code> keyword as well:<\/p>\nq=title:*SolrCloud* AND _val_:\"div(views,dislikes)\"&fq={!frange l=1}dislikes&fl=*,score<\/pre>\n{\n \"responseHeader\":{\n \"zkConnected\":true,\n \"status\":0,\n \"QTime\":50,\n \"params\":{\n \"q\":\"title:*SolrCloud* AND _val_:\\\"div(views,dislikes)\\\"\",\n \"fl\":\"*,score\",\n \"fq\":\"{!frange l=1}dislikes\",\n \"wt\":\"json\",\n \"_\":\"1592054952916\"}},\n \"response\":{\"numFound\":2,\"start\":0,\"maxScore\":201.0,\"docs\":[\n {\n \"id\":\"0818231712\",\n \"category\":[\"solr\"],\n \"title\":[\"Apache SolrCloud Example\"],\n \"published\":true,\n \"author\":[\"Kevin Yang\"],\n \"views\":2000,\n \"likes\":1000,\n \"dislikes\":10,\n \"comments\":200,\n \"publish_date\":\"2020-06-05T00:00:00Z\",\n \"_version_\":1669390419809533952,\n \"score\":201.0},\n {\n \"id\":\"0380014300\",\n \"category\":[\"solr\"],\n \"title\":[\"SolrCloud Tutorial\"],\n \"published\":true,\n \"author\":[\"Roger Goodwill\"],\n \"views\":2000,\n \"likes\":1000,\n \"dislikes\":500,\n \"comments\":10,\n \"publish_date\":\"2020-06-05T00:00:00Z\",\n \"_version_\":1669390419821068288,\n \"score\":5.0}]\n }}<\/pre>\n4.5 Using Function Query In A Sort Expression<\/h3>\n
q=title:*SolrCloud*&fq={!frange l=1}dislikes&fl=*,score&sort=div(views,dislikes) desc, score desc<\/pre>\n{\n \"responseHeader\":{\n \"zkConnected\":true,\n \"status\":0,\n \"QTime\":72,\n \"params\":{\n \"q\":\"title:*SolrCloud*\",\n \"fl\":\"*,score\",\n \"fq\":\"{!frange l=1}dislikes\",\n \"sort\":\"div(views,dislikes) desc, score desc\",\n \"wt\":\"json\",\n \"_\":\"1592061341139\"}},\n \"response\":{\"numFound\":2,\"start\":0,\"maxScore\":1.0,\"docs\":[\n {\n \"id\":\"0818231712\",\n \"category\":[\"solr\"],\n \"title\":[\"Apache SolrCloud Example\"],\n \"published\":true,\n \"author\":[\"Kevin Yang\"],\n \"views\":2000,\n \"likes\":1000,\n \"dislikes\":10,\n \"comments\":200,\n \"publish_date\":\"2020-06-05T00:00:00Z\",\n \"_version_\":1669390419809533952,\n \"score\":1.0},\n {\n \"id\":\"0380014300\",\n \"category\":[\"solr\"],\n \"title\":[\"SolrCloud Tutorial\"],\n \"published\":true,\n \"author\":[\"Roger Goodwill\"],\n \"views\":2000,\n \"likes\":1000,\n \"dislikes\":500,\n \"comments\":10,\n \"publish_date\":\"2020-06-05T00:00:00Z\",\n \"_version_\":1669390419821068288,\n \"score\":1.0}]\n }}<\/pre>\n4.6 Adding Function Results As Fields Of Documents In Search Results<\/h3>\n
q=title:*SolrCloud*&fq={!frange l=1}dislikes&fl=id,title,author,views,dislikes,score,popularity:div(views,dislikes)<\/pre>\npopularity:div(views,dislikes)<\/code> to the fields list. The output would be:<\/p>\n{\n \"responseHeader\":{\n \"zkConnected\":true,\n \"status\":0,\n \"QTime\":84,\n \"params\":{\n \"q\":\"title:*SolrCloud*\",\n \"fl\":\"id,title,author,views,dislikes,score,popularity:div(views,dislikes)\",\n \"fq\":\"{!frange l=1}dislikes\",\n \"wt\":\"json\",\n \"_\":\"1592061341139\"}},\n \"response\":{\"numFound\":2,\"start\":0,\"maxScore\":1.0,\"docs\":[\n {\n \"id\":\"0818231712\",\n \"title\":[\"Apache SolrCloud Example\"],\n \"author\":[\"Kevin Yang\"],\n \"views\":2000,\n \"dislikes\":10,\n \"popularity\":200.0,\n \"score\":1.0},\n {\n \"id\":\"0380014300\",\n \"title\":[\"SolrCloud Tutorial\"],\n \"author\":[\"Roger Goodwill\"],\n \"views\":2000,\n \"dislikes\":500,\n \"popularity\":4.0,\n \"score\":1.0}]\n }}<\/pre>\n5. Download the Sample Configuration and Data Files<\/h2>\n