基于Flume+Log4j+Kafka的日志采集架构方案(4)

其实并不复杂,注意其中有一些获取不到的信息,用?代替了,保留字段的目的在于与log4j 2.x 的日志格式完全一致,配置log4j.properties如下对接 Kafka:

log4j.rootLogger=INFO,console log4j.logger.com.banksteel.log.demo.log4j=DEBUG,kafka log4j.appender.kafka=kafka.producer.KafkaLog4jAppender log4j.appender.kafka.topic=server_log log4j.appender.kafka.brokerList=Kafka-01:9092,Kafka-02:9092,Kafka-03:9092 log4j.appender.kafka.compressionType=none log4j.appender.kafka.syncSend=true log4j.appender.kafka.layout=com.banksteel.log.demo.log4j.JsonLayout
# appender console log4j.appender.console=org.apache.log4j.ConsoleAppender log4j.appender.console.target=System.out log4j.appender.console.layout=org.apache.log4j.PatternLayout log4j.appender.console.layout.ConversionPattern=%d [%-5p] [%t] - [%l] %m%n 

通过打印日志我们可以看到其输出的最终格式如下:

{ "timeMillis": 1462713132695, "thread": "main", "level": "ERROR", "loggerName": "com.banksteel.log.demo.log4j.Demo", "source": { "method": "main", "class": "com.banksteel.log.demo.log4j.Demo", "file": "Demo.java", "line": 20 }, "endOfBatch": false, "loggerFqcn": "org.slf4j.impl.Log4jLoggerAdapter", "message": "处理业务逻辑的时候发生一个错误……", "thrown": [ { "class": "com.intellij.rt.execution.application.AppMain", "file": "AppMain.java", "line": 144, "method": "main", "location": "?", "version": "?" }, { "class": "com.intellij.rt.execution.application.AppMain", "file": "AppMain.java", "line": 144, "method": "main", "location": "?", "version": "?" }, { "class": "com.intellij.rt.execution.application.AppMain", "file": "AppMain.java", "line": 144, "method": "main", "location": "?", "version": "?" }, { "class": "com.intellij.rt.execution.application.AppMain", "file": "AppMain.java", "line": 144, "method": "main", "location": "?", "version": "?" }, { "class": "com.intellij.rt.execution.application.AppMain", "file": "AppMain.java", "line": 144, "method": "main", "location": "?", "version": "?" }, { "class": "com.intellij.rt.execution.application.AppMain", "file": "AppMain.java", "line": 144, "method": "main", "location": "?", "version": "?" } ] }

测试类:

package com.banksteel.log.demo.log4j; import org.slf4j.Logger; import org.slf4j.LoggerFactory; /** * @author 热血BUG男 * @version 1.0.0 * @since Created by gebug on 2016/5/8. */ public class Demo { private static final Logger logger = LoggerFactory.getLogger(Demo.class); public static void main(String[] args) { logger.info("输出信息……"); logger.trace("随意打印……"); logger.debug("调试信息……"); logger.warn("警告信息……"); try { throw new Exception("错误消息啊"); } catch (Exception e) { logger.error("处理业务逻辑的时候发生一个错误……", e); } } } 

log4j 2.x 与 Kafka 集成

log4j 2.x 天生支持 JsonLayout,并且与 Kafka 集成方便,我们只需要按部就班的配置一下就好了,POM.xml如下:

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/34e1564505a6b7a1c6395fcee32dc310.html