准备环境

我用的时两台服务器 一台 1g1核 一台 2g2核并且每台都装了Docker和Docker-compose

本人Linux环境

系统镜像: CentOS 7版本
服务器1: Vultr
服务器2: 阿里云轻量

准备工作

在搭建集群之前,我们需要开放我们两台服务器的端口,防止两方不能互通

1
2
3
4
5
# zookeeper 服务器开启端口 
firewall-cmd --zone=public --add-port=2181/tcp --add-port=2182/tcp --add-port=2183/tcp --permanent

#Kafka服务器开发端口
firewall-cmd --zone=public --add-port=9092/tcp --add-port=9093/tcp --add-port=9094/tcp --permanent

第一台用来搭建zookeeper集群环境

docker-compose 脚本配置准备工作

networks

我使用的是自定义的网关命令如下

docker network create --subnet 192.66.0.0/16 --gateway 192.66.0.1 --driver bridge zkk

下载如下文件或者复制一下内容

docker-compose.yml

  • docker-compose.yml 内容如下
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
version: '3.4'
networks:
  zkk:
    name: zkk
services:
  zk1:
    image: zookeeper
    restart: always
    hostname: zk1
    container_name: zk1
    networks:
      - zkk
    ports:
      - 2181:2181
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=zk1:2888:3888;2181 server.2=zk2:2888:3888;2181 server.3=zk3:2888:3888;2181
    volumes:
      - /opt/zookeeper/zk1:/data
      - /opt/zookeeper/zk1:/logs
  zk2:
    image: zookeeper
    restart: always
    hostname: zk2
    container_name: zk2
    ports:
      - 2182:2181
    networks:
      - zkk
    environment:
      ZOO_MY_ID: 2
      ZOO_SERVERS: server.1=zk1:2888:3888;2181 server.2=zk2:2888:3888;2181 server.3=zk3:2888:3888;2181
    volumes:
      - /opt/zookeeper/zk2:/data
      - /opt/zookeeper/zk2:/logs
  zk3:
    image: zookeeper
    restart: always
    hostname: zk3
    container_name: zk3
    ports:
      - 2183:2181
    networks:
      - zkk
    environment:
      ZOO_MY_ID: 3
      ZOO_SERVERS: server.1=zk1:2888:3888;2181 server.2=zk2:2888:3888;2181 server.3=zk3:2888:3888;2181
    volumes:
      - /opt/zookeeper/zk3:/data
      - /opt/zookeeper/zk3:/logs

执行脚本

1
2
# 因为我们的文件名是 docker-compose.yml 所有不需要指定文件名 如果你的文件名不是默认要求可以使用 docker -f 文件名 up 来启动
docker-compose up

Kafka 环境启动

Kafka脚本文件

docker-compose-kafka.yml

文件内容

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
version: '2.8'
networks:
  zkk:
    name: zkk
services:
  kafka:
    image: wurstmeister/kafka
    hostname: kafka1
    container_name: k1
    networks:
      - zkk
    ports:
      - 9092:9092
    environment:
      KAFKA_BROKER_ID: 0
      KAFKA_ADVERTISED_HOST_NAME: 你的Kakfa服务器ip地址(your Kafka server ip address)
      KAFKA_ZOOKEEPER_CONNECT: 你的zookeeper服务器地址:2181,你的zookeeper服务器地址:2182,你的zookeeper服务器地址:2183
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_HEAP_OPTS: "-Xmx256M -Xms128M"
  kafka2:
    image: wurstmeister/kafka
    hostname: kafka2
    container_name: k2
    ports:
      - 9093:9093
    networks:
      - zkk
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ADVERTISED_HOST_NAME: 你的Kakfa服务器ip地址(your Kafka server ip address)
      KAFKA_ZOOKEEPER_CONNECT: 你的zookeeper服务器地址:2181,你的zookeeper服务器地址:2182,你的zookeeper服务器地址:2183
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:9093
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9093
      KAFKA_HEAP_OPTS: "-Xmx256M -Xms128M"
  kafk3:
    image: wurstmeister/kafka
    hostname: kafka3
    container_name: k3
    ports:
      - 9094:9094
    networks:
      - zkk
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ADVERTISED_HOST_NAME: 你的Kakfa服务器ip地址(your Kafka server ip address)
      KAFKA_ZOOKEEPER_CONNECT: 你的zookeeper服务器地址:2181,你的zookeeper服务器地址:2182,你的zookeeper服务器地址:2183
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka3:9094
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9094
      KAFKA_HEAP_OPTS: "-Xmx256M -Xms128M"

执行docker-compose up命令

1
docker-compose -f docker-compose docker-compose-kafka.yml up

哦对了! 容器中我们的软件对应的地址
zookeeper /apache-zookeeper-3.7.0-bin

Kafka /opt/kafka