SolrCloud – 2 Nodes Solr, 1 Node ZK Setup


By: Susheel Kumar | August 3, 2015

 

Here I am going to talk about a basic SolrCloud setup on 2 separate machines (or 2 Solr nodes sitting on different machines) with 1 Zookeeper instance for development purpose.

I hardly found any article which talks about setting up 2 nodes solr instance with 1 ZK node on separate machines (can be VM), so putting this together.

The default SolrCloud script with (-e cloud) https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud  is a good start to setup SolrCloud but it is limited to setup on single machine with Solr running on different ports.

Below are the steps to setup Zookeeper & Solr Nodes.

Zookeeper Setup on Node 1/ Machine 1

  1. Download Zookeeper 3.4.6 and extracts on Node 1 / machine 1 under a directory say solrcloud.
  2. Screen Shot 2015-07-18 at 7.41.37 PM
  3. Let’s first setup Zookeeper. Go inside <ZK_HOME>/conf directory
  4. Make a copy of zoo_sample.cfg & rename to zoo.cfg (or mv zoo_sample.cfg to zoo.cfg)
  5. Edit zoo.cfg and modify data_dir parameter to a directory location where you would like Zookeeper to store its data (Similar to below where created data directory under conf for simplicity)
    • dataDir=<ZK_HOME>/conf/data

  6. Now start Zookeeper with command
    • ./bin/zkServer.sh start
  7. That’s all needs to be done for ZK at this moment.

 Solr Setup on Node 1 / Machine 1 

  1. Download solr-5.x.x & extract under directory  solrcloud as shown above
  2. Now lets create solr home directory for our setup. You can choose any location but in below example i am using solr5.x.x/server/solr/node1/solr/.
  3. Copy default zoo.cfg & solr.xml from solr5.x.x/server/solr to solr5.x.x/server/solr/node1/solr/
  4. Now lets start Solr using below command (basically you want to start in cloud mode with Zookeeper)
    • ./bin/solr start -cloud -s solr-5.x.x/server/solr/node1/solr  -p 8983 -z <Node1 IP>:2181 -m 2g

Solr Setup on Node 2 / Machine 2 

Follow exactly similar steps as above on node 2 / machine 2 except

  1. we don’t need to download & setup zookeeper since its already been setup on Node 1 &
  2. we need to point to <Node 1 IP> not node 2 when starting Solr in cloud mode. To be clear run this command
    • ./bin/solr start -cloud -s solr-5.x.x/server/solr/node1/solr  -p 8983 -z <Node1 IP>:2181 -m 2g

Upload configs to Zookeeper

  1. Before we upload the configuration files to ZK, lets create a directory to hold them on Node1.
  2. Assuming you are aware of “configsets”, lets keep your solr conf files under configsets directory for better management.  So copy your files to solr5.x.x/server/solr/configsets/<renametoyourname_configs>/conf
  3. Since we want to have Zookeeper manage our configuration, lets upload our configs to Zookeeper by below command.  (NOTE: don’t confuse zkCli below with zkCli found under ZK_HOME/bin directory. Both are different)
    •   ./server/scripts/cloud-scripts/zkcli.sh -zkhost <Node1 IP>:2181 -cmd upconfig -confname renametoyourname_configs -confdir solr5.x.x/server/solr/configsets/<renametoyourname_configs>/conf

Creating a collection

Now the final step is to create a collection using below command

http://<Node1 IP>:8983/solr/admin/collections?action=CREATE&name=<myCollection>&numShards=2&replicationFactor=2&maxShardsPerNode=2&collection.configName= renametoyourname_configs  (Remember whatever name you used to upload configs to Zookeeper, please use same name here)

Launch Solr

Go to browser & launch http://<Node1 IP>:8983/solr/#/~cloud & you shall see cloud up & running.

You will see the data getting distributed between shards once you ingest  data. For querying data on SolrCloud, i hope you can follow this article https://cwiki.apache.org/confluence/display/solr/Distributed+Requests

Please free free to contact me in case of any queries/questions/corrections.

This post has been viewed 14,257 times

3 thoughts on “SolrCloud – 2 Nodes Solr, 1 Node ZK Setup

  1. Praveen Gunja

    Hi,
    After doing lot of search on this i found this article , indeed a fresh one written for the latest apache solr 5.2.1.
    This is exactly what i am looking for , how ever i got stuck and unsuccessful in setting up this.

    Can you please elaborate the same steps more in context to windows servers?

    Thanks,
    Praveen

    Reply
    1. Susheel Kumar Post author

      Sure, Praveen. I am not sure what exact issue you are facing but in terms of windows the solr cloud setup should be fairly similar. Let me go thru some exceptions

      ZK Setup
      – Download ZK and setup on one of the two machines you may have for two nodes or if you third extra machine, you can configure there as well.
      – Setup Zookeeper similar to the steps under ZK setup section except you will have windows paths like forward slash (/) and you can use full path like “c:\solrcloud\zookeeper-3.4.6\conf\data” when setting dataDir parameter in zoo.cfg.
      – Use zkServer.cmd instead zkServer.sh since you are on windows not unix

      Setup Solr
      – Download Solr5.2.1 on both machines
      – Follow similar steps as outlined above for setting up Solr except use solr.cmd

      Upload Config to Zk
      – follow steps as above except use zkcli.bat than zkcli.sh

      Creating a collection
      – This will remain same for windows as this you will execute on your browser.

      Reply
  2. Praveen Gunja

    Hi Susheel,
    Thanks for the response. I wish you keep your blog updated regularly with the useful tips.

    By the way coming to the solrcloud, I am trying with VM machines created locally in my laptop . I will follow the steps again , in case any issues will get back to you

    Thanks,
    Praveen

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>