Linux环境 solr-7.4.0安装【一篇就够】

一、环境说明

三台安装了CenOS7操作系统的虚拟机

master（192.168.11.128）、slave1（192.168.11.129）、slave2（192.168.11.30）

二、下载solr安装包

地址：http://mirrors.shu.edu.cn/apache/lucene/solr/7.4.0/solr-7.4.0.zip

三、解压solr安装包

[[email protected] usr]# unzip solr-7.4.0.zip

四、配置环境变量

[[email protected] usr]# vi /etc/profile
编辑profile文件末尾追加：

# solr environment
export SOLR_INSTALL_HOME=/usr/solr-7.4.0
export PATH=$PATH:$SOLR_INSTALL_HOME/bin

执行如下命令使环境变量生效

[[email protected] usr]# source /etc/profile

五、以SolrCloud模式启动solr

[[email protected] usr]# solr start -c
*** [WARN] *** Your open file limit is currently 1024.  
 It should be set to 65000 to avoid operational disruption. 
 If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false in your profile or solr.in.sh
*** [WARN] ***  Your Max Processes Limit is currently 3818. 
 It should be set to 65000 to avoid operational disruption. 
 If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to false in your profile or solr.in.sh
WARNING: Starting Solr as the root user is a security risk and not considered best practice. Exiting.
         Please consult the Reference Guide. To override this check, start with argument '-force'

将/usr/solr-7.4.0/bin/solr.in.sh文件中，SOLR_ULIMIT_CHECKS设置为false，消除WARN

# Settings for common system values that may cause operational imparement when system defaults are used.
# Solr can use many processes and many file handles. On modern operating systems the savings by leaving
# these settings low is minuscule, while the consequence can be Solr instability. To turn these checks off, set
# SOLR_ULIMIT_CHECKS=false either here or as part of your profile.

# Different limits can be set in solr.in.sh or your profile if you prefer as well.
#SOLR_RECOMMENDED_OPEN_FILES=
#SOLR_RECOMMENDED_MAX_PROCESSES=
SOLR_ULIMIT_CHECKS=false

再次启动（根据WARNING提示因为是root用户，需要加 -force）

[[email protected] bin]# solr start -c -force
NOTE: Please install lsof as this script needs it to determine if Solr is listening on port 8983.

Started Solr server on port 8983 (pid=2772). Happy searching!

[[email protected] bin]# 
 

六、访问solr页面

需要先执行命令（systemctl stop firewalld.service）关闭防火墙，否则页面不能正常打开

http://192.168.11.129:8983/solr/#/

七、扩展-配置solr.in.sh

     51 # Set the ZooKeeper connection string if using an external ZooKeeper ensemble
     52 # e.g. host1:2181,host2:2181/chroot
     53 # Leave empty if not using SolrCloud
     54 ZK_HOST="master:2181,slave1:2181,slave2:2181"     <==前提：zookeeper安装完毕，安装在各个机器上的solr通过
     55                                                       ZK共享collection信息，即一台服务器上创建好collection之后，其它服务器可见
     56 # Set the ZooKeeper client timeout (for SolrCloud mode)
     57 #ZK_CLIENT_TIMEOUT="15000"
     58 
     59 # By default the start script uses "localhost"; override the hostname here
     60 # for production SolrCloud environments to control the hostname exposed to cluster state
     61 SOLR_HOST="slave1"        <==solr安装所在机器名/IP地址
     62 
     63 # By default Solr will try to connect to Zookeeper with 30 seconds in timeout; override the timeout if needed
     64 #SOLR_WAIT_FOR_ZK="30"
     65 
     66 # By default the start script uses UTC; override the timezone if needed
     67 #SOLR_TIMEZONE="UTC"
     68 SOLR_TIMEZONE="UTC+8"

     89 # Path to a directory for Solr to store cores and their data. By default, Solr will use server/solr
     90 # If solr.xml is not stored in ZooKeeper, this directory needs to contain solr.xml
     91 SOLR_HOME=/usr/solr-7.4.0/solr_home            <==需先创建相应目录
     92 
     93 # Path to a directory that Solr will use as root for data folders for each core.
     94 # If not set, defaults to <instance_dir>/data. Overridable per core through 'dataDir' core property
     95 SOLR_DATA_HOME=/usr/solr-7.4.0/solr_data_home            <==需先创建相应目录
     96 
     97 # Solr provides a default Log4J configuration xml file in server/resources
     98 # however, you may want to customize the log settings and file appender location
     99 # so you can point the script to use a different log4j2.xml file
    100 #LOG4J_PROPS=/var/solr/log4j2.xml
    101 
    102 # Changes the logging level. Valid values: ALL, TRACE, DEBUG, INFO, WARN, ERROR, FATAL, OFF. Default is INFO
    103 # This is an alternative to changing the rootLogger in log4j2.xml
    104 #SOLR_LOG_LEVEL=INFO
    105 
    106 # Location where Solr should write logs to. Absolute or relative to solr start dir
    107 SOLR_LOGS_DIR=/usr/solr-7.4.0/solr_logs                   <==需先创建相应目录
    108 
    109 # Enables log rotation before starting Solr. Setting SOLR_LOG_PRESTART_ROTATION=true will let Solr take care of pre
    110 # start rotation of logs. This is false by default as log4j2 handles this for us. If you choose to use another log
    111 # framework that cannot do startup rotation, you may want to enable this to let Solr rotate logs on startup.
    112 SOLR_LOG_PRESTART_ROTATION=true             <==每次重启启动solr都会新生成一个log文件
    113 
    114 # Sets the port Solr binds to, default is 8983
    115 SOLR_PORT=8983
 

各solr服务器通过zookeeper共享的内容如下：

[[email protected] solr_logs]# zkCli.sh             <==登录zookeeper
log4j:WARN No such property [maxFileSize] in org.apache.log4j.DailyRollingFileAppender.
Connecting to localhost:2181
Welcome to ZooKeeper!
JLine support is enabled

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /
[configs, zookeeper, overseer, aliases.json, live_nodes, collections, overseer_elect, security.json, clusterstate.json, autoscaling, autoscaling.json]
[zk: localhost:2181(CONNECTED) 1] 
solr页面也可以看到，如下：

 

 

八、扩展-配置solr.in.sh后，再次以SolrCloud模式启动
启动时需要保证solr_home下有solr.xml文件，否则会失败。solr.xml内容如下：

     29 <solr>
     30 
     31   <solrcloud>
     32 
     33     <str name="host">${host:slave1}</str>
     34     <int name="hostPort">${jetty.port:8983}</int>
     35     <str name="hostContext">${hostContext:solr}</str>
     36 
     37     <bool name="genericCoreNodeNames">${genericCoreNodeNames:true}</bool>
     38 
     39     <int name="zkClientTimeout">${zkClientTimeout:30000}</int>
     40     <int name="distribUpdateSoTimeout">${distribUpdateSoTimeout:600000}</int>
     41     <int name="distribUpdateConnTimeout">${distribUpdateConnTimeout:60000}</int>
     42     <str name="zkCredentialsProvider">${zkCredentialsProvider:org.apache.solr.common.cloud.DefaultZkCredentialsProvider}</str>
     43     <str name="zkACLProvider">${zkACLProvider:org.apache.solr.common.cloud.DefaultZkACLProvider}</str>
     44 
     45   </solrcloud>
     46 
     47   <shardHandlerFactory name="shardHandlerFactory"
     48     class="HttpShardHandlerFactory">
     49     <int name="socketTimeout">${socketTimeout:600000}</int>
     50     <int name="connTimeout">${connTimeout:60000}</int>
     51   </shardHandlerFactory>
     52 
     53 </solr>

九、扩展-以SolrCloud模式启动后，创建/删除collection

[[email protected] solr-7.4.0]# solr create_collection -c collection1 -shards 2 -replicationFactor 2 -p 8983 -force
WARNING: Using _default configset with data driven schema functionality. NOT RECOMMENDED for production use.
         To turn off: bin/solr config -c collection1 -p 8983 -action set-user-property -property update.autoCreateFields -value false
INFO  - 2018-09-02 23:31:02.105; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
Created collection 'collection1' with 2 shard(s), 2 replica(s) with config-set 'collection1'

默认会把 /usr/solr-7.4.0/server/solr/configsets/_default/conf 下内容，上传到zookeeper 的 /configs/collection1 目录下

[[email protected] conf]# pwd
/usr/solr-7.4.0/server/solr/configsets/_default/conf
[[email protected] conf]# ls
lang  managed-schema  params.json  protwords.txt  solrconfig.xml  stopwords.txt  synonyms.txt

[[email protected] solr_logs]# zkCli.sh                     <==登录zookeeper
log4j:WARN No such property [maxFileSize] in org.apache.log4j.DailyRollingFileAppender.
Connecting to localhost:2181
Welcome to ZooKeeper!
JLine support is enabled

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /configs/collection1
[managed-schema, protwords.txt, solrconfig.xml, synonyms.txt, stopwords.txt, lang, params.json]
[zk: localhost:2181(CONNECTED) 1] 
 

删除collection1，同时zookeeper上对应的配置文件目录（/configs/collection1）也会被删除

[[email protected] conf]# solr delete -c collection1
INFO  - 2018-09-02 23:46:26.788; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
{
  "responseHeader":{
    "status":0,
    "QTime":1364},
  "success":{
    "master:8983_solr":{"responseHeader":{
        "status":0,
        "QTime":264}},
    "slave2:8983_solr":{"responseHeader":{
        "status":0,
        "QTime":274}},
    "slave1:8983_solr":{"responseHeader":{
        "status":0,
        "QTime":370}}}}

Deleted collection 'collection1' using command:
http://slave2:8983/solr/admin/collections?action=DELETE&name=collection1

zookeeper上对应Node不再存在

[zk: localhost:2181(CONNECTED) 1] ls /configs/collection1
Node does not exist: /configs/collection1

 

从solr服务器上传配置文件到zookeeper（方法一）

[[email protected] conf]# solr zk                   <==查看solr zk所有命令

ERROR: Zookeeper operation (one of 'upconfig', 'downconfig', 'rm', 'mv', 'cp', 'ls', 'mkroot') is required!

  Usage: solr zk upconfig|downconfig -d <confdir> -n <configName> [-z zkHost]
         solr zk cp [-r] <src> <dest> [-z zkHost]
         solr zk rm [-r] <path> [-z zkHost]
         solr zk mv <src> <dest> [-z zkHost]
         solr zk ls [-r] <path> [-z zkHost]
         solr zk mkroot <path> [-z zkHost]

[[email protected] conf]# solr zk upconfig -d /usr/solr-7.4.0/solr_confdir/coll1_conf/conf -n collection1
INFO  - 2018-09-03 00:15:42.578; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
Uploading /usr/solr-7.4.0/solr_confdir/coll1_conf/conf for config collection1 to ZooKeeper at master:2181,slave1:2181,slave2:2181

从solr服务器上传配置文件到zookeeper（方法二）

sh /usr/solr-7.4.0/server/scripts/cloud-scripts/zkcli.sh -zkhost master:2181,slave1:2181,slave2:2181 -cmd upconfig -confdir /usr/solr-7.4.0/solr_confdir/coll1_conf/conf（solr服务器conf目录） -confname collection1（zookeeper对应collection目录）

 

从zookeeper上下载配置文件到solr服务器（方法一）

[[email protected] coll1_conf]# solr zk downconfig -d /usr/solr-7.4.0/solr_confdir/coll1_conf -n collection1
INFO  - 2018-09-03 00:09:08.594; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
Downloading configset collection1 from ZooKeeper at master:2181,slave1:2181,slave2:2181 to directory /usr/solr-7.4.0/solr_confdir/coll1_conf/conf
[[email protected] coll1_conf]# ls
conf
[[email protected] coll1_conf]# cd conf
[[email protected] conf]# ls
lang  managed-schema  params.json  protwords.txt  solrconfig.xml  stopwords.txt  synonyms.txt
[[email protected] conf]# 

从zookeeper上下载配置文件到solr服务器（方法二）
sh /usr/solr-7.4.0/server/scripts/cloud-scripts/zkcli.sh -cmd downconfig -zkhost master:2181 -confdir /usr/solr-7.4.0/solr_confdir/coll1_conf/conf（solr服务器conf目录） -confname collection1（zookeeper对应collection目录）

 

十、扩展-以SolrCloud模式启动后，将MySQL中的表数据import到collection中

需要在配置文件solrconfig.xml中增加requestHandler，否则点击Dataimport会显示：

The solrconfig.xml file for this index does not have an operational DataImportHandler defined.

1，修改配置文件solrconfig.xml（增加蓝字部分）

     60               <lib dir="./lib" />
     61 
     62   <!-- A 'dir' option by itself adds any files found in the directory
     63        to the classpath, this is useful for including all jars in a
     64        directory.
     65 
     66        When a 'regex' is specified in addition to a 'dir', only the
     67        files in that directory which completely match the regex
     68        (anchored on both ends) will be included.
     69 
     70        If a 'dir' option (with or without a regex) is used and nothing
     71        is found that matches, a warning will be logged.
     72 
     73        The examples below can be used to load some solr-contribs along
     74        with their external dependencies.
     75     -->
     76   <lib dir="${solr.install.dir:../../../..}/contrib/extraction/lib" regex=".*\.jar" />
     77   <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-cell-\d.*\.jar" />
     78 
     79   <lib dir="${solr.install.dir:../../../..}/contrib/clustering/lib/" regex=".*\.jar" />
     80   <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-clustering-\d.*\.jar" />
     81 
     82   <lib dir="${solr.install.dir:../../../..}/contrib/langid/lib/" regex=".*\.jar" />
     83   <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-langid-\d.*\.jar" />
     84 
     85   <lib dir="${solr.install.dir:../../../..}/contrib/velocity/lib" regex=".*\.jar" />
     86   <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-velocity-\d.*\.jar" />
     87 
     88   <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-dataimporthandler-.*\.jar" />
     89         <lib dir="${solr.install.dir:../../../..}/contrib/dataimporthandler-extras/lib/" regex=".*\.jar" />
     90   <!-- an exact 'path' can be used instead of a 'dir' to specify a
     91        specific jar file.  This will cause a serious error to be logged
     92        if it can't be loaded.
     93     -->
    694     <requestHandler name="/dataimport" class="solr.DataImportHandler">
    695     <lst name="defaults">
    696       <str name="config">solr-data-config.xml</str>
    697     </lst>
    698   </requestHandler>
 

2，增加solr-data-config.xml配置文件，内容如下：

<dataConfig>
        <dataSource name="mysqlDB" type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
                url="jdbc:mysql://192.168.0.4:3306/test?useUnicode=true&amp;characterEncoding=utf8"
                user="root" password="root"/>
  <document>
    <entity name="films" dataSource="mysqlDB"
      query="select id,name,directed_by,genre,initial_release_date from films">
      <field column="id" name="id"/>
      <field column="name" name="name"/>
      <field column="directed_by" name="directed_by"/>
      <field column="genre" name="genre"/>
      <field column="initial_release_date" name="initial_release_date"/>
    </entity>
  </document>
</dataConfig>

3，修改配置文件managed-schema（增加蓝字部分）

    431   <field name="_root_" type="string" docValues="false" indexed="true" stored="false"/>
    432   <field name="_text_" type="text_general" multiValued="true" indexed="true" stored="false"/>
    433   <field name="_version_" type="plong" indexed="false" stored="false"/>
    434   <field name="directed_by" type="text_general" indexed="true" stored="true"/>
    435   <field name="id" type="string" multiValued="false" indexed="true" required="true" stored="true"/>     <==默认存在
    436   <field name="name" type="text_general" indexed="true" stored="true"/>
 

4，将mysql-connector-java-5.1.44.jar放到/usr/solr-7.4.0/server/solr-webapp/webapp/WEB-INF/lib下

5，使用root用户访问mysql可能会发生错误： Host 'SJMZ' is not allowed to connect to this MySQL server

这个错误，是因为MySQL不允许远程登录，所以远程登录失败了，解决方法如下：
a) 登录mysql
b) 执行use mysql;
c) 执行update user set host = '%' where user = 'root';
d) 执行FLUSH PRIVILEGES
注: d)步是刷新MySQL的权限相关表，一定不要忘了。

6，solr页面执行Dataimport

7，solr执行Query即可查看到导入数据