Apache Solr and Drupal
Apache Solr is an open source search platform that can make search queries much faster. It is surprisingly easy to integrate it with Drupal by using the Search API framework.
Complex searches on big amount of data can be slow, and caching is not always the ultimate answer. If you are using facets with Facet API + Search API with Database Search as a backend this is often the case. The more entities are indexed, the slower the queries are, and there are probably too many possible filters for effective caching solutions.
Meet Apache Solr
Apache Solr is an open source search platform that can make these kind of queries much faster. It stores data in a way that is very fast for queries required by faceted search. It's good to know that Solr is also very good for serving full text searches and scales well. In theory it could work as a general storage engine, but adding and updating data requires rebuilding the index, which is very slow (compared to any SQL systems). Additionally, Solr can't guarantee transactional consistency. This makes it a bad general storage engine, but a good search platform.
How to install
Install Search API and create an index. If you are not sure how to do it, check out my earlier blogpost. Download and install Search Api Solr. Check the documentation for supported Apache Solr versions and download the newest. On Linux, assuming you have an installed JRE extract Solr files then
$ cd [solr]/example # Replace [solr] with your solr home directory.
$ cp [drupal_home]/sites/all/modules/contrib/search_api_solr/solr/conf/4.x/* [solr]/example/solr/collection1
# Example is a pre-configured server for solr. It's ok to use it now. Same for collection1 which is an example dataset. $ java -jar start.jar
# Solr server is now listening on port 8983. You can check the admin interface by typing localhost:8983 in a browser.
On other systems, check the corresponding tutorials.
Integration with Search API
On your Drupal site, go to Configuration > Search and metadata > Search API and add a new server. The class should be Solr Service. Now you can associate an index with this and after indexing the data, all searches of the index will be served by Apache Solr. You can find more tricks for Solr here and here. Do not forget to secure the server before using it in production.
Today most of the websites have search functionality. With the help of Apache Solr the time spent on waiting for a search result can be radically reduced. In this article we are going to set up a basic searching infrastructure on a *nix-based system.
What do you do when you need to search in files as well? For a recent project I had to enable users to search the content of attached files mainly in .pdf format. The Apache Solr with Tika seemed to be a good solution.