一、安装插件

在elasticsearch安装模块

[root@k8s-master elasticsearch]# ./bin/elasticsearch-plugin install file:///usr/share/elasticsearch/ingest-user-agent-6.6.0.zip
-> Downloading file:///usr/share/elasticsearch/ingest-user-agent-6.6.0.zip
[=================================================] 100%   
-> Installed ingest-user-agent
[root@k8s-master elasticsearch]# ./bin/elasticsearch-plugin install file:///usr/share/elasticsearch/ingest-geoip-6.6.0.zip 
-> Downloading file:///usr/share/elasticsearch/ingest-geoip-6.6.0.zip
[=================================================] 100%   

systemctl restart elasticsearch

二、修改filebeat配置文件

在末尾添加

 filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true 
  reload.period: 10s

使用命令查看

[root@k8s-node2 filebeat]# filebeat modules list
Enabled:
nginx

Disabled:
 

开启模块的2种方式

1、将 /etc/filebeat/modules.d/nginx.yml.disabled  改名为nginx.yml就可以了。

2、

filebeat enable nginx 

修改配置文件

[root@k8s-node2 filebeat]# cat modules.d/nginx.yml
- module: nginx
  # Access logs
  access:
    enabled: true
    var.paths: ["/var/log/nginx/*.log"]

  # Error logs
  error:
    enabled: true
    var.paths: ["/var/log/nginx/error.log"]

 

 

[root@k8s-node2 filebeat]# cat filebeat.yml
output.elasticsearch:
  hosts: ["10.0.0.11:9200"]
  indices:
    - index: "nginx-access-%{[beat.version]}-%{+yyyy.MM}"
      when.contains:
       tags: "access"
    - index: "nginx-error-%{[beat.version]}-%{+yyyy.MM}"
      when.contains:
       tags: "error"

setup.template.name: "nginx"
setup.template.pattern: "nginx-*"
setup.template.enabled: false
setup.template.overwrite: true

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true 
  reload.period: 10s
 

三、查看nginx日志

 [root@k8s-node2 filebeat]# cat /var/log/nginx/access.log 
10.0.0.13 - - [08/Nov/2020:14:56:33 +0800] "GET / HTTP/1.1" 200 4833 "-" "curl/7.29.0" "-"
 

ok没问题。

四、清空日志、重启filebeat

rm -rf /var/log/nginx/*

systemctl restart nignx

systemctl restart filebeat

创建kibana的时候error日志的时候选择read_timestamp