Zookeeper 服务器端和客户端扩大节点数据1M大小限制

Zookeeper中每个结点默认的数据量上限是1M，如果需要存入大于1M的数据量，则要修改jute.maxbuffer参数，先来看下zookeeper中这个参数的意义

jute.maxbuffer： 默认值1048575，单位字节，用于配置单个数据节点（ZNode）上可以存储的最大数据大小。需要注意的是，在修改该参数的时候，需要在zookeeper集群的所有服务端以及客户端上设置才能生效。

接下来，我们就来看看，怎么设置这个参数吧。

服务器端：
如果服务器端没有修改该参数，直接写入过大数据会抛错：org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /test
Zookeeper 服务器端和客户端扩大节点数据1M大小限制

解决办法：

zkServer.sh 新增-Djute.maxbuffer配置，这边以10M为例，具体大小需按实际情况修改（ZOO_USER_CFG为修改过部分的关键词）：

 #!/usr/bin/env bash
 
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
 # The ASF licenses this file to You under the Apache License, Version 2.0
 # (the "License"); you may not use this file except in compliance with
 # the License.  You may obtain a copy of the License at
 #
 #     http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
 #
 # If this scripted is run out of /usr/bin or some other system bin directory
 # it should be linked to and not copied. Things like java jar files are found
 # relative to the canonical path of this script.
 #
 
 
 
 # use POSTIX interface, symlink is followed automatically
 ZOOBIN="${BASH_SOURCE-$0}"
 ZOOBIN="$(dirname "${ZOOBIN}")"
 ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
 
 if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
   . "$ZOOBINDIR/../libexec/zkEnv.sh"
 else
   . "$ZOOBINDIR/zkEnv.sh"
 fi
 
 # See the following page for extensive details on setting
 # up the JVM to accept JMX remote management:
 # http://java.sun.com/javase/6/docs/technotes/guides/management/agent.html
 # by default we allow local JMX connections
 
 ZOO_USER_CFG="-Djute.maxbuffer=10240000"
 
 if [ "x$JMXLOCALONLY" = "x" ]
 then
     JMXLOCALONLY=false
 fi
 
 if [ "x$JMXDISABLE" = "x" ] || [ "$JMXDISABLE" = 'false' ]
 then
   echo "ZooKeeper JMX enabled by default" >&2
   if [ "x$JMXPORT" = "x" ]
   then
     # for some reason these two options are necessary on jdk6 on Ubuntu
     #   accord to the docs they are not necessary, but otw jconsole cannot
     #   do a local attach
     ZOOMAIN="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=$JMXLOCALONLY org.apache.zookeeper.server.quorum.QuorumPeerMain"
   else
     if [ "x$JMXAUTH" = "x" ]
     then
       JMXAUTH=false
     fi
     if [ "x$JMXSSL" = "x" ]
     then
       JMXSSL=false
     fi
     if [ "x$JMXLOG4J" = "x" ]
     then
       JMXLOG4J=true
     fi
     echo "ZooKeeper remote JMX Port set to $JMXPORT" >&2
     echo "ZooKeeper remote JMX authenticate set to $JMXAUTH" >&2
     echo "ZooKeeper remote JMX ssl set to $JMXSSL" >&2
     echo "ZooKeeper remote JMX log4j set to $JMXLOG4J" >&2
     ZOOMAIN="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=$JMXPORT -Dcom.sun.management.jmxremote.authenticate=$JMXAUTH -Dcom.sun.management.jmxremote.ssl=$JMXSSL -Dzookeeper.jmx.log4j.disable=$JMXLOG4J org.apache.zookeeper.server.quorum.QuorumPeerMain"
   fi
 else
     echo "JMX disabled by user request" >&2
     ZOOMAIN="org.apache.zookeeper.server.quorum.QuorumPeerMain"
 fi
 
 if [ "x$SERVER_JVMFLAGS"  != "x" ]
 then
     JVMFLAGS="$SERVER_JVMFLAGS $JVMFLAGS"
 fi
 
 if [ "x$2" != "x" ]
 then
     ZOOCFG="$ZOOCFGDIR/$2"
 fi
 
 # if we give a more complicated path to the config, don't screw around in $ZOOCFGDIR
 if [ "x$(dirname "$ZOOCFG")" != "x$ZOOCFGDIR" ]
 then
     ZOOCFG="$2"
 fi
 
 if $cygwin
 then
     ZOOCFG=`cygpath -wp "$ZOOCFG"`
     # cygwin has a "kill" in the shell itself, gets confused
     KILL=/bin/kill
 else
     KILL=kill
 fi
 
 echo "Using config: $ZOOCFG" >&2
 
 case "$OSTYPE" in
 *solaris*)
   GREP=/usr/xpg4/bin/grep
   ;;
 *)
   GREP=grep
   ;;
 esac
 if [ -z "$ZOOPIDFILE" ]; then
     ZOO_DATADIR="$($GREP "^[[:space:]]*dataDir" "$ZOOCFG" | sed -e 's/.*=//')"
     if [ ! -d "$ZOO_DATADIR" ]; then
         mkdir -p "$ZOO_DATADIR"
     fi
     ZOOPIDFILE="$ZOO_DATADIR/zookeeper_server.pid"
 else
     # ensure it exists, otw stop will fail
     mkdir -p "$(dirname "$ZOOPIDFILE")"
 fi
 
 if [ ! -w "$ZOO_LOG_DIR" ] ; then
 mkdir -p "$ZOO_LOG_DIR"
 fi
 
 _ZOO_DAEMON_OUT="$ZOO_LOG_DIR/zookeeper.out"
 
 case $1 in
 start)
     echo  -n "Starting zookeeper ... "
     if [ -f "$ZOOPIDFILE" ]; then
       if kill -0 `cat "$ZOOPIDFILE"` > /dev/null 2>&1; then
          echo $command already running as process `cat "$ZOOPIDFILE"`. 
          exit 0
       fi
     fi
     nohup "$JAVA" "$ZOO_USER_CFG" "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" \
     -cp "$CLASSPATH" $JVMFLAGS $ZOOMAIN "$ZOOCFG" > "$_ZOO_DAEMON_OUT" 2>&1 < /dev/null &
     if [ $? -eq 0 ]
     then
       case "$OSTYPE" in
       *solaris*)
         /bin/echo "${!}\\c" > "$ZOOPIDFILE"
         ;;
       *)
         /bin/echo -n $! > "$ZOOPIDFILE"
         ;;
       esac
       if [ $? -eq 0 ];
       then
         sleep 1
         echo STARTED
       else
         echo FAILED TO WRITE PID
         exit 1
       fi
     else
       echo SERVER DID NOT START
       exit 1
     fi
     ;;
 start-foreground)
     ZOO_CMD=(exec "$JAVA")
     if [ "${ZOO_NOEXEC}" != "" ]; then
       ZOO_CMD=("$JAVA")
     fi
     "${ZOO_CMD[@]}" "$ZOO_USER_CFG" "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" \
     -cp "$CLASSPATH" $JVMFLAGS $ZOOMAIN "$ZOOCFG"
     ;;
 print-cmd)
     echo "\"$JAVA\" -Dzookeeper.log.dir=\"${ZOO_LOG_DIR}\" -Dzookeeper.root.logger=\"${ZOO_LOG4J_PROP}\" -cp \"$CLASSPATH\" $JVMFLAGS $ZOOMAIN \"$ZOOCFG\" > \"$_ZOO_DAEMON_OUT\" 2>&1 < /dev/null"
     ;;
 stop)
     echo -n "Stopping zookeeper ... "
     if [ ! -f "$ZOOPIDFILE" ]
     then
       echo "no zookeeper to stop (could not find file $ZOOPIDFILE)"
     else
       $KILL -9 $(cat "$ZOOPIDFILE")
       rm "$ZOOPIDFILE"
       echo STOPPED
     fi
     exit 0
     ;;
 upgrade)
     shift
     echo "upgrading the servers to 3.*"
     "$JAVA" "$ZOO_USER_CFG" "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" \
     -cp "$CLASSPATH" $JVMFLAGS org.apache.zookeeper.server.upgrade.UpgradeMain ${@}
     echo "Upgrading ... "
     ;;
 restart)
     shift
     "$0" stop ${@}
     sleep 3
     "$0" start ${@}
     ;;
 status)
     # -q is necessary on some versions of linux where nc returns too quickly, and no stat result is output
     clientPortAddress=`$GREP "^[[:space:]]*clientPortAddress[^[:alpha:]]" "$ZOOCFG" | sed -e 's/.*=//'`
     if ! [ $clientPortAddress ]
     then
 	clientPortAddress="localhost"
     fi
     clientPort=`$GREP "^[[:space:]]*clientPort[^[:alpha:]]" "$ZOOCFG" | sed -e 's/.*=//'`
     STAT=`"$JAVA" "$ZOO_USER_CFG" "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" \
              -cp "$CLASSPATH" $JVMFLAGS org.apache.zookeeper.client.FourLetterWordMain \
              $clientPortAddress $clientPort srvr 2> /dev/null    \
           | $GREP Mode`
     if [ "x$STAT" = "x" ]
     then
         echo "Error contacting service. It is probably not running."
         exit 1
     else
         echo $STAT
         exit 0
     fi
     ;;
 *)
     echo "Usage: $0 {start|start-foreground|stop|restart|status|upgrade|print-cmd}" >&2
 
 esac

客户端：
改完zkServer.sh重启zookeeper后超过1M数据可以写进去了，但是从客户端读的时候又抛错了java.io.IOException: Unreasonable length = 1560001
Zookeeper 服务器端和客户端扩大节点数据1M大小限制

zkCli.sh

接下来就要去改客户端的配置，如果是直接使用zookeeper自带的zkCli.sh读取数据，那么要在zkCli.sh中新增-Djute.maxbuffer配置，这边以10M为例，具体大小需按实际情况修改，（ZOO_USER_CFG为修改过部分的关键词）：

 #!/usr/bin/env bash
 
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
 # The ASF licenses this file to You under the Apache License, Version 2.0
 # (the "License"); you may not use this file except in compliance with
 # the License.  You may obtain a copy of the License at
 #
 #     http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
 #
 # This script cleans up old transaction logs and snapshots
 #
 
 #
 # If this scripted is run out of /usr/bin or some other system bin directory
 # it should be linked to and not copied. Things like java jar files are found
 # relative to the canonical path of this script.
 #
 
 # use POSTIX interface, symlink is followed automatically
 ZOOBIN="${BASH_SOURCE-$0}"
 ZOOBIN="$(dirname "${ZOOBIN}")"
 ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
 ZOO_USER_CFG="-Djute.maxbuffer=10240000"
 
 if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
   . "$ZOOBINDIR"/../libexec/zkEnv.sh
 else
   . "$ZOOBINDIR"/zkEnv.sh
 fi
 
 "$JAVA" "$ZOO_USER_CFG" "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" \
      -cp "$CLASSPATH" $CLIENT_JVMFLAGS $JVMFLAGS \
      org.apache.zookeeper.ZooKeeperMain "[email protected]"

代码块中

如果是在代码中读取，那么要把修改后的jute.maxbuffer属性值重新写入，我这边是在java代码中使用zookeeper-3.4.10.jar包，先来看下这个包中是怎么初始化jute.maxbuffer值的
Zookeeper 服务器端和客户端扩大节点数据1M大小限制
maxbuffer会先从系统属性中读取jute.maxbuffer值（System.getProperty()）,如果取不到则赋值1048575
知道maxbuffer的取值逻辑后，我们就可以对症下药啦，只要在初始化zookeeper客户端之前，往系统属性中加入修改后的jute.maxbuffer键值对就可以啦
Zookeeper 服务器端和客户端扩大节点数据1M大小限制
最后再啰嗦一点，zookeeper上不宜存储太多数据，所以jute.maxbuffer的值需要按具体情况定哦

参考：
《从Paxos到ZooKeeper》
https://blog.****.net/hopingwhite/article/details/8255979

Zookeeper 服务器端和客户端扩大节点数据1M大小限制

相关推荐