服务器集群怎么搭建（Graylog4.2集群部署完整教程）

GrayLog多节点集群架构如下
 
(图片可点击放大查看)
下面介绍Graylog4.2集群部署过程
基础环境准备
三台服务器：CentOS7.9的虚拟机
/data分区（LVM) 用于Elasticsearch日志数据存储
内存大小为6GB
IP地址和主机名如下：

192.168.31.211 graylog01 graylog01.walkingcloud.cn
192.168.31.212 graylog02 graylog02.walkingcloud.cn
192.168.31.213 graylog03 graylog03.walkingcloud.cn

均已关闭SELINUX
 
(图片可点击放大查看)
 
(图片可点击放大查看)
一、配置MongoDB集群
1、三台均安装mongodb

cat > /etc/yum.repos.d/mongodb-org.repo << \EOF
[mongodb-org]
name=MongoDB Repository
baseurl=https://mirrors.aliyun.com/mongodb/yum/redhat/$releasever/mongodb-org/4.4/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.4.asc
EOF

yum install -y mongodb-org

 
(图片可点击放大查看)
2、三台均先启动mongodb

systemctl daemon-reload
systemctl enable mongod.service
systemctl start mongod.service
systemctl –type=service –state=active | grep mongod
firewall-cmd –add-port=27017/tcp –permanent –zone=public
firewall-cmd –reload

 
(图片可点击放大查看)
3、主节点生成授权认证keyfile文件并拷贝到其它节点

openssl rand -base64 756 > /var/lib/mongo/access.keyfile
chown mongod:mongod /var/lib/mongo/access.keyfile
chmod 600 /var/lib/mongo/access.keyfile
scp -rp /var/lib/mongo/access.keyfile root@graylog02:/var/lib/mongo/
scp -rp /var/lib/mongo/access.keyfile root@graylog03:/var/lib/mongo/

 
(图片可点击放大查看)
 
(图片可点击放大查看)
4、vi修改/etc/mongod.conf
以主节点为例，修改配置文件

vi /etc/mongod.conf
net:
port: 27017
bindIp: 192.168.31.211

security:
keyFile: /var/lib/mongo/access.keyfile

replication:
replSetName: graylog-rs

 
(图片可点击放大查看)
graylog02修改成如下
 
(图片可点击放大查看)
graylog03修改成如下
 
(图片可点击放大查看)
5、初始化集群
这里为了本地能登录mongo

先将bindIp: 192.168.31.211修改为bindIp: 0.0.0.0
并重启服务
systemctl restart mongod.service

 
输入mongo进数据库

use admin

rs.initiate( {
_id : "graylog-rs",
members: [
{ _id: 0, host: "192.168.31.211:27017" },
{ _id: 1, host: "192.168.31.212:27017" },
{ _id: 2, host: "192.168.31.213:27017" }
]
})

rs.status()查看集群状态

 
(图片可点击放大查看)
6、创建graylog数据库并设置密码

#修改admin用户密码
use admin
db.createUser({user: "admin", pwd: "Admin@2021", roles: ["root"]})
db.auth("admin","Admin@2021")
#创建graylog数据库并设置密码
use graylog
db.createUser({
user: "graylog",
pwd: "Graylog2021",
"roles" : [{
"role" : "dbOwner",
"db" : "graylog"
}, {
"role" : "readWrite",
"db" : "graylog"
}]
})

 
(图片可点击放大查看)
7、将主节点bindIp配置恢复并重启mongod服务
将graylog01的bindIp修改成192.168.31.211并重启服务
这时使用账号和密码登录mongo
 
(图片可点击放大查看)
在graylog02 重启mongod服务，使graylog01重新变为Primary角色
二、搭建Elasticsearch集群
1、三台节点均以rpm包方式安装Elasticsearch

cd /opt
wget https://mirrors.cloud.tencent.com/elasticstack/yum/elastic-7.x/7.16.2/elasticsearch-7.16.2-x86_64.rpm

rpm -ivh elasticsearch-7.16.2-x86_64.rpm

systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service
firewall-cmd –add-port=9200/tcp –permanent –zone=public
firewall-cmd –reload

 
(图片可点击放大查看)
2、创建存储和日志文件夹

mkdir -p /data/elasticsearch/data
mkdir -p /data/elasticsearch/logs
chown -R elasticsearch:elasticsearch /data/elasticsearch

cp /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml_default

 
(图片可点击放大查看)
3、修改elasticsearch.yml配置文件

vi/etc/elasticsearch/elasticsearch.yml

1)graylog01节点
#添加并修改成如下行

cluster.name: graylog-cluster
action.auto_create_index: false
node.name: graylog01
node.master: true
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
network.host: 192.168.31.211
http.port: 9200
transport.port: 9300
discovery.seed_hosts: ["192.168.31.211:9300", "192.168.31.212:9300", "192.168.31.213:9300"]
cluster.initial_master_nodes: ["graylog01"]

 
 
(图片可点击放大查看)
2)graylog02节点

cat /etc/elasticsearch/elasticsearch.yml | grep -v "^#" | grep -v "^$"

cluster.name: graylog-cluster
action.auto_create_index: false
node.name: graylog02
node.master: false
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
network.host: 192.168.31.212
http.port: 9200
transport.port: 9300
discovery.seed_hosts: ["192.168.31.211:9300", "192.168.31.212:9300", "192.168.31.213:9300"]
3)graylog03节点

cat /etc/elasticsearch/elasticsearch.yml | grep -v "^#" | grep -v "^$"
cluster.name: graylog-cluster
action.auto_create_index: false
node.name: graylog03
node.master: false
node.data: true
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
network.host: 192.168.31.213
http.port: 9200
transport.port: 9300
discovery.seed_hosts: ["192.168.31.211:9300", "192.168.31.212:9300", "192.168.31.213:9300"]
4、三个节点均修改jvm.options配置文件上内存大小

firewall-cmd –add-port=9300/tcp –permanent –zone=public
firewall-cmd –reload
vim /etc/elasticsearch/jvm.options
设置jvm内存大小为物理内存的一半
并重启elasticsearch.service
systemctl restart elasticsearch.service

 
(图片可点击放大查看)
 
(图片可点击放大查看)
5、查看elasticsearch集群状态

curl -s -XGET 'http://192.168.31.211:9200/_cluster/health?pretty=true'
curl -s -XGET 'http://192.168.31.211:9200/_cat/nodes?v'

 
(图片可点击放大查看)
三、安装GraylogServer并配置GraylogServer集群
1、安装jdk，pwgen和graylog-server

yum install -y java-1.8.0-openjdk-headless.x86_64
yum install -y pwgen

 
(图片可点击放大查看)
 
(图片可点击放大查看)

rpm -ivh https://packages.graylog2.org/repo/packages/graylog-4.2-repository_latest.rpm
yum install graylog-server -y

 
(图片可点击放大查看)

systemctl enable graylog-server
systemctl start graylog-server

cp /etc/graylog/server/server.conf /etc/graylog/server/server.conf_default

pwgen -N 1 -s 96
XC0Epiv5SnWFdm82nsUWAJN3t2MHaFEaSFHd6RLPf1nzxwnmubT0n7NQdrK8jCDEOS05DtrkGHDDE61490OUJKBOOXIAT4LI

echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1
Enter Password: Graylog@2021
10dfabb9595634675701865aa1c6e774d89d59f4a104ab128fbffcdaa3cf8f7b

 
(图片可点击放大查看)
2、修改graylog的主配置文件server.conf
主节点graylog01的配置如下

/etc/graylog/server/server.conf
cat /etc/graylog/server/server.conf | grep -v "^#" | grep -v "^$"
is_master = true
node_id_file = /etc/graylog/server/node-id
password_secret = XC0Epiv5SnWFdm82nsUWAJN3t2MHaFEaSFHd6RLPf1nzxwnmubT0n7NQdrK8jCDEOS05DtrkGHDDE61490OUJKBOOXIAT4LI
root_password_sha2 = 10dfabb9595634675701865aa1c6e774d89d59f4a104ab128fbffcdaa3cf8f7b
root_timezone = Asia/Shanghai
bin_dir = /usr/share/graylog-server/bin
data_dir = /var/lib/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = 192.168.31.211:9000
http_publish_uri = http://192.168.31.211:9000/
elasticsearch_hosts = http://graylog01:9200,http://graylog02:9200,http://graylog03:9200
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = true
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 8
outputbuffer_processors = 16
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://graylog:Graylog2021@graylog01:27017,graylog02:27017,graylog03:27017/graylog?replicaSet=graylog-rs
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
proxied_requests_thread_pool_size = 32

 
(图片可点击放大查看)
3、重启graylog-server服务，并开放9000 web端口
建议将配置文件scp到graylog02和graylog03上
 
(图片可点击放大查看)

firewall-cmd –add-port=9000/tcp –permanent –zone=public
firewall-cmd –reload
systemctl restart graylog-server

 
(图片可点击放大查看)
graylog02,graylog03节点修改如下三处即可

is_master = true
http_bind_address = 192.168.31.213:9000
http_publish_uri = http://192.168.31.213:9000/

 
(图片可点击放大查看)
登录Graylog主界面查看GrayLog集群情况
1、查看Nodes信息
 
(图片可点击放大查看)
2、查看Cluster信息
 
(图片可点击放大查看)
3、添加Input，可以设置成全局的Input
记得三台几点防火墙开放Input设置的端口，例如syslog1514

firewall-cmd –add-port=1514/udp –permanent –zone=public
firewall-cmd –reload

 
(图片可点击放大查看)
 
(图片可点击放大查看)
 
(图片可点击放大查看)
Tips：修改graylog的jvm内存大小
 
(图片可点击放大查看)
 
(图片可点击放大查看)
本文参考如下链接完成

https://docs.graylog.org/v1/docs/multinode-setup

https://docs.mongodb.com/manual/tutorial/deploy-replica-set/

https://docs.mongodb.com/manual/tutorial/deploy-replica-set-with-keyfile-access-control/#std-label-deploy-repl-set-with-auth

https://cloud.tencent.com/developer/article/1615815

https://zhuanlan.zhihu.com/p/120698020

https://www.cnblogs.com/opsdemo/p/15035379.html举报评论 7