Most of the applications must have a some kind of a search function. The problem is that search functions are often huge resource hogs and they can kill the performance of our application by causing heavy load to the database. That is why transferring that load to an external search server is a great idea.<\/p>\n
This is the first part of my Spring Data Solr tutorial. During this tutorial we will implement a search function to a todo application which is the example application of my Spring MVC Test tutorial<\/a>.<\/p>\n The requirements of our search function are simple. It must return a list of todo entries which title or description contains the used search term. The search result Before we can start the implementation of our search function, we need to take a look at the Solr search server. This blog entry provides us the basic information about Solr and it is divided into three parts:<\/p>\n Let\u2019s get started.<\/p>\n Let\u2019s start by getting a brief introduction to the Solr search server. This introduction is very thin and it provides only the information we need to know in order to understand the implementation of our search function.<\/p>\n Also, even though Solr<\/a> is heavily dependent from Lucene<\/a>, this blog entry makes no difference between them.<\/p>\n This section describes<\/p>\n An index<\/em> consists of documents<\/em> which are essentially a collection of fields<\/em>. If we compare this data model to the data model of a relational database, we notice the following similarities:<\/p>\n Each field of a document can be either indexed<\/em>, stored<\/em> or both. The meaning of these terms is described in the following:<\/p>\n When a new document is added to Solr, indexed and stored fields are processed in a different way. This difference is described in the following:<\/p>\n The search function can be divided into three steps which are described in the following:<\/p>\n The schema is used to configure the following things:<\/p>\n The schema is configured in a file called schema.xml<\/em>, and we can create a schema for our application by following these steps:<\/p>\n These steps are described with more details in the following subsections. The skeleton schema of our Solr instance looks as follows:<\/p>\n Note<\/strong>: This section describes the schema of the example application of my blog entry called Running Solr with Maven<\/a>.<\/p>\n A field type specifies the following things:<\/p>\n We can configure a field type by using the fieldType<\/em> element. Its attributes which are used in our schema are described in the following:<\/p>\n We can configure the field types of our schema by following these steps: These steps are described with more details in the following.<\/p>\n Let\u2019s start by configuring a simple field type for long fields which are not analyzed on the index or search phase. We can configure this field type by following these steps:<\/p>\n Our field type declaration looks as follows:<\/p>\n The next step is to configure a simple field type for string fields which are not analyzed on the index or search phase. We can configure this field type by following these steps:<\/p>\n The declaration of our string field looks as follows:<\/p>\n The last step is to configure the text_general<\/em> field type. We can do this by following these steps:<\/p>\n The declaration of the text_general<\/em> field type looks as follows:<\/p>\n We can get more information about the analysis phase by reading the following documents:<\/p>\n We can add new fields to our document by adding field<\/em> elements to the schema.xml<\/em> file. The field<\/em> element has many attributes<\/a> but at this point we need to understand the meaning of following attributes:<\/p>\n In order to make maximize the performance of our Solr instance, we must follow the following guidelines:<\/p>\n We can get more information about the optimal field configuration by reading the following documents:<\/p>\n We are now ready to configure the actual fields of our schema. Let\u2019s first talk a bit about the information which we need show on the search result page. This information is described in the following:<\/p>\n When we know that our application must be able to search the contents of the title and description of a todo entry, we can add the required fields to our schema by following these steps:<\/p>\n Our field declarations looks as follows:<\/p>\n We use copy fields<\/a> to copy the contents of title<\/em> and description<\/em> fields to the text<\/em> field. We can configure the copy fields by using the copyField<\/em> element. We can create the required configuration by following these steps:<\/p>\n The declaration of our copy fields looks as follows:<\/p>\n The unique key<\/a> is a field which should be unique for all documents. It is not mandatory to specify the unique key of a document but if we decide to do so, it means that the index cannot contain two documents which both have same value in the field configured as the unique key.<\/p>\n In our case, we use the field \u2018id\u2019 as the unique key of our document. We can make this configuration by adding the following XML to the schema.xml<\/em> file:<\/p>\n Solr offers a REST-like HTTP API which we can use to add information to Solr and execute search queries against its index. Both of these use cases are described in the following.<\/p>\n Note<\/strong>: This section assumes that we are using the example application of my blog entry called Running Solr with Maven<\/a>.<\/p>\n We can add new information to Solr by following these steps:<\/p>\n The contents of our request body looks as follows:<\/p>\n We have now added two documents to our Solr index by using the REST-like API provided by Solr.<\/p>\n However, it is good to know that there are other options which we can use to add information to the Solr index. These options are described in the following documents:<\/p>\n We are now ready to search the information stored in the index of our Solr instance. We can execute search queries against the Solr index by following these guidelines:<\/p>\n Let\u2019s move on and find out how we can list all documents found from the index and execute a simple search query against the indexed data.<\/p>\n We can list all documents in JSON format by following these steps:<\/p>\n When we send a GET request to the url \u2018http:\/\/localhost:8983\/solr\/todo\/select?q=*%3A*&wt=json\u2019, we should receive the following JSON:<\/p>\n We can search all documents which title or description contains the word \u2018application\u2019 by following these steps:<\/p>\n When we send a GET request tot he url \u2018http:\/\/localhost:8983\/solr\/todo\/select?q=application&wt=json\u2019, we should receive the following JSON:<\/p>\n We have now obtained the information which is required to understand the concepts described in the next parts of my Spring Data Solr tutorial. This blog entry has teached us three things:<\/p>\n <\/p>\n Most of the applications must have a some kind of a search function. The problem is that search functions are often huge resource hogs and they can kill the performance of our application by causing heavy load to the database. That is why transferring that load to an external search server is a great idea. …<\/p>\n","protected":false},"author":429,"featured_media":80,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[470,30,321],"class_list":["post-12865","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-enterprise-java","tag-apache-solr","tag-spring","tag-spring-data"],"yoast_head":"\n
\n
\npage must also provide a link to the page used to view the information of a todo entry.<\/p>\n\n
A Brief Introduction to Solr<\/h2>\n
\n
The Data Model<\/h3>\n
\n
\n
Adding Information to the Index<\/h3>\n
\n
Searching Information from the Index<\/h3>\n
\n
Creating the Schema<\/h2>\n
\n
\n
<?xml version=\"1.0\" encoding=\"UTF-8\" ?>\r\n<schema name=\"todo\" version=\"1.5\">\r\n <fields>\r\n <!-- Configure fields here -->\r\n <\/fields>\r\n\r\n <!-- Configure unique key -->\r\n <!-- Configure copy fields here -->\r\n \r\n <types>\r\n <!-- Configure field types here -->\r\n <\/types>\r\n<\/schema><\/pre>\n
Configuring the Field Types<\/h3>\n
\n
\n
\n
Configuring the Field Type for Long Fields<\/h2>\n
\n
<fieldType name=\"long\" class=\"solr.TrieLongField\" precisionStep=\"0\" positionIncrementGap=\"0\"\/><\/pre>\n
Configuring the Field Type for String Fields<\/h2>\n
\n
<fieldType name=\"string\" class=\"solr.StrField\" sortMissingLast=\"true\" \/><\/pre>\n
Configuring the Field Type for Fields Containing Text<\/h2>\n
\n
<fieldType name=\"text_general\" class=\"solr.TextField\" positionIncrementGap=\"100\">\r\n <!-- Configures the analysis done at the index phase -->\r\n <analyzer type=\"index\">\r\n <!-- Uses word break rules of the Unicode Text Segmentation algorith when splitting text into words. -->\r\n <tokenizer class=\"solr.StandardTokenizerFactory\"\/>\r\n <!-- Removes words found from stopwords.txt file. This filter is case insensitive. -->\r\n <filter class=\"solr.StopFilterFactory\" ignoreCase=\"true\" words=\"stopwords.txt\" enablePositionIncrements=\"true\" \/>\r\n <!-- Transforms text to lower case -->\r\n <filter class=\"solr.LowerCaseFilterFactory\"\/>\r\n <\/analyzer>\r\n <!-- Configures the analysis done at the query time -->\r\n <analyzer type=\"query\">\r\n <!-- Uses word break rules of the Unicode Text Segmentation algorith when splitting text into words. -->\r\n <tokenizer class=\"solr.StandardTokenizerFactory\"\/>\r\n <!-- Removes words found from stopwords.txt file. This filter is case insensitive. -->\r\n <filter class=\"solr.StopFilterFactory\" ignoreCase=\"true\" words=\"stopwords.txt\" enablePositionIncrements=\"true\" \/>\r\n <!-- Applies synonyms found from the synonyms.txt file. -->\r\n <filter class=\"solr.SynonymFilterFactory\" synonyms=\"synonyms.txt\" ignoreCase=\"true\" expand=\"true\"\/>\r\n <!-- Transforms text to lower case -->\r\n <filter class=\"solr.LowerCaseFilterFactory\"\/>\r\n <\/analyzer>\r\n<\/fieldType><\/pre>\n
\n
Configuring the Fields of Our Document<\/h3>\n
\n
\n
\n
\n
\n
<field name=\"id\" type=\"string\" indexed=\"true\" stored=\"true\" required=\"true\" multiValued=\"false\" \/>\r\n<field name=\"title\" type=\"text_general\" indexed=\"true\" stored=\"true\" required=\u201ctrue\u201d multiValued=\"false\"\/>\r\n<field name=\"description\" type=\"text_general\" indexed=\"true\" stored=\"false\" multiValued=\"false\"\/>\r\n<field name=\"text\" type=\"text_general\" indexed=\"true\" stored=\"false\" multiValued=\"true\"\/>\r\n<field name=\"_version_\" type=\"long\" indexed=\"true\" stored=\"true\"\/><\/pre>\n
Configuring the Copy Fields<\/h3>\n
\n
<copyField source=\"title\" dest=\"text\"\/>\r\n<copyField source=\"description\" dest=\"text\"\/><\/pre>\n
Configuring the Unique Key Field of Our Document<\/h3>\n
<uniqueKey>id<\/uniqueKey><\/pre>\n
Using the REST-like HTTP API<\/h2>\n
Adding Information to Solr<\/h3>\n
\n
[\r\n {\r\n \"id\":\"1\",\r\n \"title\":\"Write introduction to Solr\",\r\n \"description\":\"This blog entry provides an introduction to Solr search server\"\r\n },\r\n {\r\n \"id\":\"2\",\r\n \"title\":\"Implement example application\",\r\n \"description\":\"This application demonstrates the usage of spring-data-solr.\"\r\n }\r\n]<\/pre>\n\n
Searching Information from Solr Index<\/h3>\n
\n
Finding All Documents of the Index<\/h2>\n
\n
{\r\n \"responseHeader\": {\r\n \"status\":0,\r\n \"QTime\":1,\r\n \"params\":{\r\n \"wt\":\"json\",\r\n \"q\":\"*:*\"\r\n }\r\n },\r\n \"response\":{\r\n \"numFound\":2,\r\n \"start\":0,\r\n \"docs\":[\r\n {\r\n \"id\":\"1\",\r\n \"title\":\"Write introduction to Solr\",\r\n \"_version_\":1425949176574771200\r\n },\r\n {\r\n \"id\":\"2\",\r\n \"title\":\"Implement example application\",\r\n \"_version_\":1425949176662851584\r\n }\r\n ]\r\n }\r\n}<\/pre>\nSearching Information from the Index<\/h2>\n
\n
{\r\n \"responseHeader\":{\r\n \"status\":0,\r\n \"QTime\":7,\r\n \"params\":{\r\n \"wt\":\"json\",\r\n \"q\":\"application\"\r\n }\r\n },\r\n \"response\":{\r\n \"numFound\":1,\r\n \"start\":0,\r\n \"docs\":[\r\n {\r\n \"id\":\"2\",\r\n \"title\":\"Implement example application\",\r\n \"_version_\":1425949176662851584\r\n }\r\n ]\r\n }\r\n}<\/pre>\nThe End<\/h2>\n
\n