Presto详细安装和测试--单节点和集群模式
参考:
https://prestodb.github.io/docs/current/installation/deployment.html
https://www.imooc.com/video/19187
presto server下载地址:https://repo1.maven.org/maven2/com/facebook/presto/presto-server/0.226/
presto client下载地址:https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.226/
环境:
presto-server-0.226(要求java版本151+)
jdk1.8.0_221
单节点模式:
coordinator和work在同一个节点
etc/config.properties
If you are setting up a single machine for testing that will function as both a coordinator and worker, use this configuration:
coordinator=true node-scheduler.include-coordinator=false http-server.http.port=9999 query.max-memory=50GB query.max-memory-per-node=5GB query.max-total-memory-per-node=10GB discovery-server.enabled=true discovery.uri=http://192.168.0.9:9999
注意:如果query.max-memory-per-node配置的过小,会在查询的时候导致OOM,导致presto server服务对应的worker挂掉
etc/node.properties
node.environment=production node.id=presto1 node.data-dir=/data/presto
node.environment需要保证每个节点相同
node.id需要保证每个节点的唯一性
node.data-dir为presto运行日志目录,运行日志在${node.data-dir}/var/log目录下
etc/jvm.config
-server -Xmx16G -XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:+UseGCOverheadLimit -XX:+ExplicitGCInvokesConcurrent -XX:+HeapDumpOnOutOfMemoryError -XX:+ExitOnOutOfMemoryError
etc/log.properties
com.facebook.presto=INFO
etc/catalog/hive.properties
参考:https://prestodb.github.io/docs/current/connector/hive.html#hive-configuration-properties
hive.metastore.uri为hive的metastore对应的thift地址,
可以查看hive的hive-site.xml配置文件查询相应配置。
connector.name=hive-hadoop2 hive.metastore.uri=thrift://192.168.0.5:9083 hive.config.resources=${HADOOP_HOME}/etc/hadoop/conf/core-site.xml,${HADOOP_HOME}/etc/hadoop/conf/hdfs-site.xml
配置完成后,启动./bin/launcher start
通过jps,
可以看到PrestoServer已经启动, 如果未启动,可以查看presto的日志,查找原因
验证:
可以看到有一个worker启动了
测试:
启动客户端:
下载相应版本的客户端jar(https://repo1.maven.org/maven2/com/facebook/presto/presto-cli/0.226/)
./presto-cli-0.226-executable.jar --server 192.168.0.9:9999 --catalog hive --schema default
schema为对应的数据库实例
2.集群模式
集群模式唯一的区别在于worker节点的etc/config.properties
配置如下
coordinator=false
http-server.http.port=9999
query.max-memory=50GB
query.max-memory-per-node=5GB
query.max-total-memory-per-node=10GB
discovery.uri=http://192.168.0.9:9999
启动worker节点
./launcher start
可以看到workers已经变为2了