SolrCloud - 2 Nodes Solr, 1 Node ZK Setup
Here I am going to talk about a basic SolrCloud setup on 2 separate machines (or 2 Solr nodes sitting on different machines) with 1 Zookeeper instance for development purpose.
I hardly found any article which talks about setting up 2 nodes solr instance with 1 ZK node on separate machines (can be VM), so putting this together.
The default SolrCloud script with (-e cloud) https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud is a good start to setup SolrCloud but it is limited to setup on single machine with Solr running on different ports.
Below are the steps to setup Zookeeper & Solr Nodes.
Zookeeper Setup on Node 1/ Machine 1
- Download Zookeeper 3.4.6 and extracts on Node 1 / machine 1 under a directory say solrcloud.
- Let’s first setup Zookeeper. Go inside <ZK_HOME>/conf directory
- Make a copy of zoo_sample.cfg & rename to zoo.cfg (or mv zoo_sample.cfg to zoo.cfg)
- Edit zoo.cfg and modify data_dir parameter to a directory location where you would like Zookeeper to store its data (Similar to below where created data directory under conf for simplicity)
dataDir=<ZK_HOME>/conf/data
- Now start Zookeeper with command
- ./bin/zkServer.sh start
- That’s all needs to be done for ZK at this moment.
Solr Setup on Node 1 / Machine 1
- Download solr-5.x.x & extract under directory solrcloud as shown above
- Now lets create solr home directory for our setup. You can choose any location but in below example i am using solr5.x.x/server/solr/node1/solr/.
- Copy default zoo.cfg & solr.xml from solr5.x.x/server/solr to solr5.x.x/server/solr/node1/solr/
- Now lets start Solr using below command (basically you want to start in cloud mode with Zookeeper)
./bin/solr start -cloud -s solr-5.x.x/server/solr/node1/solr -p 8983 -z <Node1 IP>:2181 -m 2g
Solr Setup on Node 2 / Machine 2
Follow exactly similar steps as above on node 2 / machine 2 except
- we don’t need to download & setup zookeeper since its already been setup on Node 1 &
- we need to point to <Node 1 IP> not node 2 when starting Solr in cloud mode. To be clear run this command
- ./bin/solr start -cloud -s solr-5.x.x/server/solr/node1/solr -p 8983 -z <Node1 IP>:2181 -m 2g
Upload configs to Zookeeper
- Before we upload the configuration files to ZK, lets create a directory to hold them on Node1.
- Assuming you are aware of “configsets”, lets keep your solr conf files under configsets directory for better management. So copy your files to solr5.x.x/server/solr/configsets/<renametoyourname_configs>/conf
- Since we want to have Zookeeper manage our configuration, lets upload our configs to Zookeeper by below command. (NOTE: don’t confuse zkCli below with zkCli found under ZK_HOME/bin directory. Both are different)
- ./server/scripts/cloud-scripts/zkcli.sh -zkhost <Node1 IP>:2181 -cmd upconfig -confname renametoyourname_configs -confdir solr5.x.x/server/solr/configsets/<renametoyourname_configs>/conf
Creating a collection
Now the final step is to create a collection using below command
http://<Node1 IP>:8983/solr/admin/collections?action=CREATE&name=<myCollection>&numShards=2&replicationFactor=2&maxShardsPerNode=2&collection.configName= renametoyourname_configs (Remember whatever name you used to upload configs to Zookeeper, please use same name here)
Launch Solr
Go to browser & launch http://<Node1 IP>:8983/solr/#/~cloud & you shall see cloud up & running.
You will see the data getting distributed between shards once you ingest data. For querying data on SolrCloud, i hope you can follow this article https://cwiki.apache.org/confluence/display/solr/Distributed+Requests
Please free free to contact me in case of any queries/questions/corrections.
Write a comment
- Adi February 1, 2019, 12:00 pmI am trying to find information regarding setting up highly available Solr with no single point of failures. I am just wondering if this is the write place to start. I did follow the whole setup but I am confused if I need to index for each instance on each node and each shard.reply
- Mari November 28, 2018, 11:55 amCan you please make it more clear to understand.reply
- benny November 21, 2018, 12:34 pmThank you for the post. it helped me a lot. only one thing is missing is configset copy is not fully described, if that is there ever thing is full.reply
- Praveen Gunja August 23, 2015, 9:35 pmHi Susheel, Thanks for the response. I wish you keep your blog updated regularly with the useful tips. By the way coming to the solrcloud, I am trying with VM machines created locally in my laptop . I will follow the steps again , in case any issues will get back to you Thanks, Praveenreply
- Susheel Kumar August 23, 2015, 2:44 pmSure, Praveen. I am not sure what exact issue you are facing but in terms of windows the solr cloud setup should be fairly similar. Let me go thru some exceptions ZK Setup - Download ZK and setup on one of the two machines you may have for two nodes or if you third extra machine, you can configure there as well. - Setup Zookeeper similar to the steps under ZK setup section except you will have windows paths like forward slash (/) and you can use full path like "c:\solrcloud\zookeeper-3.4.6\conf\data" when setting dataDir parameter in zoo.cfg. - Use zkServer.cmd instead zkServer.sh since you are on windows not unix Setup Solr - Download Solr5.2.1 on both machines - Follow similar steps as outlined above for setting up Solr except use solr.cmd Upload Config to Zk - follow steps as above except use zkcli.bat than zkcli.sh Creating a collection - This will remain same for windows as this you will execute on your browser.reply
- Praveen Gunja August 23, 2015, 1:15 pmHi, After doing lot of search on this i found this article , indeed a fresh one written for the latest apache solr 5.2.1. This is exactly what i am looking for , how ever i got stuck and unsuccessful in setting up this. Can you please elaborate the same steps more in context to windows servers? Thanks, Praveenreply