当前位置： 首页 > news >正文

最近在搭建ELK日志平台时，logstash报错JSON parse error

news 2025/7/6 0:24:13

直接进入正题，我在搭建elk日志，使用最简单的log4j2 socket json格式 输出到logstash.

但是logstash报错如下：

[WARN ] 2023-08-30 10:15:17.766 [nioEventLoopGroup-2-2] jsonlines - JSON parse error, original data now in message field {:message=>“incompatible json object type=java.lang.String , only hash map or arrays are supported”, :exception=>LogStash::Json::ParserError, :data=>" “message” : “Started FasterUserApplication in 1.472 seconds (JVM running for 2.232)”,\r"}

总是提示json格式有问题：
logstash配置如图：

input {tcp {port => 4560codec => json_lines}
}
output {elasticsearch {hosts => ["https://192.168.245.140:9200"]index => "applog-%{+YYYY.MM.dd}"template_name => "applog"user => "applog"password => "applog"cacert => "/usr/local/service/elasticsearch-8.9.1/config/certs/http_ca.crt"}file {path => "/usr/local/service/test.log"}
}

log4j2最开始配置如下，下称为配置1：

 <Socket name="socket" host="192.168.245.140" port="4560" protocol="TCP" ><JsonLayout properties="true" ><KeyValuePair key="project" value="demo1"></KeyValuePair></JsonLayout></Socket>

以上配置，每次有日志时，logstash就抛出json解析错误。

[WARN ] 2023-08-30 10:15:17.766 [nioEventLoopGroup-2-2] jsonlines - JSON parse error, original data now in message field {:message=>“incompatible json object type=java.lang.String , only hash map or arrays are supported”, :exception=>LogStash::Json::ParserError, :data=>" “message” : “Started FasterUserApplication in 1.472 seconds (JVM running for 2.232)”,\r"}

后经查证，log4j2修改配置如下，下称为配置2:

<Socket name="socket" host="192.168.245.140" port="4560" protocol="TCP" ><JsonLayout properties="true"  complete="false" compact="true" eventEol="true"locationInfo="false"includeStacktrace="true" stacktraceAsString="true"objectMessageAsJsonObject="true"><KeyValuePair key="project" value="demo1"></KeyValuePair></JsonLayout></Socket>

错误就不再出现了，下面的配置主要是多了json不再美化输出，每条日志记录输出一行json。

以下为对比:
log4j配置1,json日志输出格式为：

"C:\Program Files\Java\jdk-13.0.1\bin\java.exe" -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:10980,suspend=y,server=n -XX:TieredStopAtLevel=1 -noverify -Dspring.output.ansi.enabled=always -Dcom.sun.management.jmxremote -Dspring.jmx.enabled=true -Dspring.liveBeansView.mbeanDomain -Dspring.application.admin.enabled=true -javaagent:C:\Users\Miste\.IntelliJIdea2019.3\system\captureAgent\debugger-agent.jar -Dfile.encoding=UTF-8 -classpath "D:\workspace\playground\faster-demo\faster-user\target\classes;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot-starter-log4j2\2.5.5\spring-boot-starter-log4j2-2.5.5.jar;D:\workspace\mavenLocalRepository\org\apache\logging\log4j\log4j-slf4j-impl\2.14.1\log4j-slf4j-impl-2.14.1.jar;D:\workspace\mavenLocalRepository\org\slf4j\slf4j-api\1.7.32\slf4j-api-1.7.32.jar;D:\workspace\mavenLocalRepository\org\apache\logging\log4j\log4j-api\2.14.1\log4j-api-2.14.1.jar;D:\workspace\mavenLocalRepository\org\apache\logging\log4j\log4j-core\2.14.1\log4j-core-2.14.1.jar;D:\workspace\mavenLocalRepository\org\apache\logging\log4j\log4j-jul\2.14.1\log4j-jul-2.14.1.jar;D:\workspace\mavenLocalRepository\org\slf4j\jul-to-slf4j\1.7.32\jul-to-slf4j-1.7.32.jar;D:\workspace\mavenLocalRepository\com\lmax\disruptor\3.4.2\disruptor-3.4.2.jar;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot-starter-web\2.5.5\spring-boot-starter-web-2.5.5.jar;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot-starter\2.5.5\spring-boot-starter-2.5.5.jar;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot\2.5.5\spring-boot-2.5.5.jar;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot-autoconfigure\2.5.5\spring-boot-autoconfigure-2.5.5.jar;D:\workspace\mavenLocalRepository\jakarta\annotation\jakarta.annotation-api\1.3.5\jakarta.annotation-api-1.3.5.jar;D:\workspace\mavenLocalRepository\org\yaml\snakeyaml\1.28\snakeyaml-1.28.jar;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot-starter-json\2.5.5\spring-boot-starter-json-2.5.5.jar;D:\workspace\mavenLocalRepository\com\fasterxml\jackson\core\jackson-databind\2.12.5\jackson-databind-2.12.5.jar;D:\workspace\mavenLocalRepository\com\fasterxml\jackson\core\jackson-annotations\2.12.5\jackson-annotations-2.12.5.jar;D:\workspace\mavenLocalRepository\com\fasterxml\jackson\core\jackson-core\2.12.5\jackson-core-2.12.5.jar;D:\workspace\mavenLocalRepository\com\fasterxml\jackson\datatype\jackson-datatype-jdk8\2.12.5\jackson-datatype-jdk8-2.12.5.jar;D:\workspace\mavenLocalRepository\com\fasterxml\jackson\datatype\jackson-datatype-jsr310\2.12.5\jackson-datatype-jsr310-2.12.5.jar;D:\workspace\mavenLocalRepository\com\fasterxml\jackson\module\jackson-module-parameter-names\2.12.5\jackson-module-parameter-names-2.12.5.jar;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot-starter-tomcat\2.5.5\spring-boot-starter-tomcat-2.5.5.jar;D:\workspace\mavenLocalRepository\org\apache\tomcat\embed\tomcat-embed-core\9.0.53\tomcat-embed-core-9.0.53.jar;D:\workspace\mavenLocalRepository\org\apache\tomcat\embed\tomcat-embed-el\9.0.53\tomcat-embed-el-9.0.53.jar;D:\workspace\mavenLocalRepository\org\apache\tomcat\embed\tomcat-embed-websocket\9.0.53\tomcat-embed-websocket-9.0.53.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-web\5.3.10\spring-web-5.3.10.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-beans\5.3.10\spring-beans-5.3.10.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-webmvc\5.3.10\spring-webmvc-5.3.10.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-aop\5.3.10\spring-aop-5.3.10.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-context\5.3.10\spring-context-5.3.10.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-expression\5.3.10\spring-expression-5.3.10.jar;D:\workspace\mavenLocalRepository\mysql\mysql-connector-java\8.0.26\mysql-connector-java-8.0.26.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-core\5.3.10\spring-core-5.3.10.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-jcl\5.3.10\spring-jcl-5.3.10.jar;H:\ProframFiles\IntelliJ IDEA 2019.3\lib\idea_rt.jar" com.github.misterchangray.user.FasterUserApplication
Java HotSpot(TM) 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
Connected to the target VM, address: '127.0.0.1:0', transport: 'socket'
Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended.   ____          _            __ _ _/\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \\\/  ___)| |_)| | | | | || (_| |  ) ) ) )'  |____| .__|_| |_|_| |_\__, | / / / /=========|_|==============|___/=/_/_/_/:: Spring Boot ::                (v2.5.5){"instant" : {"epochSecond" : 1693363254,"nanoOfSecond" : 88010000},"thread" : "main","level" : "INFO","loggerName" : "com.github.misterchangray.user.FasterUserApplication","message" : "Starting FasterUserApplication using Java 13.0.1 on DESKTOP-KVEV94V with PID 13812 (D:\\workspace\\playground\\faster-demo\\faster-user\\target\\classes started by Miste in D:\\workspace\\playground\\faster-demo)","endOfBatch" : false,"loggerFqcn" : "org.apache.commons.logging.LogAdapter$Log4jLog","contextMap" : { },"threadId" : 1,"threadPriority" : 5,"project" : "demo1"
}
{"instant" : {"epochSecond" : 1693363254,"nanoOfSecond" : 167298600},"thread" : "main","level" : "DEBUG","loggerName" : "com.github.misterchangray.user.FasterUserApplication","message" : "Running with Spring Boot v2.5.5, Spring v5.3.10","endOfBatch" : false,"loggerFqcn" : "org.apache.commons.logging.LogAdapter$Log4jLog","contextMap" : { },"threadId" : 1,"threadPriority" : 5,"project" : "demo1"
}
{"instant" : {"epochSecond" : 1693363254,"nanoOfSecond" : 169298100},"thread" : "main","level" : "INFO","loggerName" : "com.github.misterchangray.user.FasterUserApplication","message" : "No active profile set, falling back to default profiles: default","endOfBatch" : false,"loggerFqcn" : "org.apache.commons.logging.LogAdapter$Log4jLog","contextMap" : { },"threadId" : 1,"threadPriority" : 5,"project" : "demo1"
}

配置2日志输出格式为：

"C:\Program Files\Java\jdk-13.0.1\bin\java.exe" -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:8338,suspend=y,server=n -XX:TieredStopAtLevel=1 -noverify -Dspring.output.ansi.enabled=always -Dcom.sun.management.jmxremote -Dspring.jmx.enabled=true -Dspring.liveBeansView.mbeanDomain -Dspring.application.admin.enabled=true -javaagent:C:\Users\Miste\.IntelliJIdea2019.3\system\captureAgent\debugger-agent.jar -Dfile.encoding=UTF-8 -classpath "D:\workspace\playground\faster-demo\faster-user\target\classes;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot-starter-log4j2\2.5.5\spring-boot-starter-log4j2-2.5.5.jar;D:\workspace\mavenLocalRepository\org\apache\logging\log4j\log4j-slf4j-impl\2.14.1\log4j-slf4j-impl-2.14.1.jar;D:\workspace\mavenLocalRepository\org\slf4j\slf4j-api\1.7.32\slf4j-api-1.7.32.jar;D:\workspace\mavenLocalRepository\org\apache\logging\log4j\log4j-api\2.14.1\log4j-api-2.14.1.jar;D:\workspace\mavenLocalRepository\org\apache\logging\log4j\log4j-core\2.14.1\log4j-core-2.14.1.jar;D:\workspace\mavenLocalRepository\org\apache\logging\log4j\log4j-jul\2.14.1\log4j-jul-2.14.1.jar;D:\workspace\mavenLocalRepository\org\slf4j\jul-to-slf4j\1.7.32\jul-to-slf4j-1.7.32.jar;D:\workspace\mavenLocalRepository\com\lmax\disruptor\3.4.2\disruptor-3.4.2.jar;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot-starter-web\2.5.5\spring-boot-starter-web-2.5.5.jar;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot-starter\2.5.5\spring-boot-starter-2.5.5.jar;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot\2.5.5\spring-boot-2.5.5.jar;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot-autoconfigure\2.5.5\spring-boot-autoconfigure-2.5.5.jar;D:\workspace\mavenLocalRepository\jakarta\annotation\jakarta.annotation-api\1.3.5\jakarta.annotation-api-1.3.5.jar;D:\workspace\mavenLocalRepository\org\yaml\snakeyaml\1.28\snakeyaml-1.28.jar;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot-starter-json\2.5.5\spring-boot-starter-json-2.5.5.jar;D:\workspace\mavenLocalRepository\com\fasterxml\jackson\core\jackson-databind\2.12.5\jackson-databind-2.12.5.jar;D:\workspace\mavenLocalRepository\com\fasterxml\jackson\core\jackson-annotations\2.12.5\jackson-annotations-2.12.5.jar;D:\workspace\mavenLocalRepository\com\fasterxml\jackson\core\jackson-core\2.12.5\jackson-core-2.12.5.jar;D:\workspace\mavenLocalRepository\com\fasterxml\jackson\datatype\jackson-datatype-jdk8\2.12.5\jackson-datatype-jdk8-2.12.5.jar;D:\workspace\mavenLocalRepository\com\fasterxml\jackson\datatype\jackson-datatype-jsr310\2.12.5\jackson-datatype-jsr310-2.12.5.jar;D:\workspace\mavenLocalRepository\com\fasterxml\jackson\module\jackson-module-parameter-names\2.12.5\jackson-module-parameter-names-2.12.5.jar;D:\workspace\mavenLocalRepository\org\springframework\boot\spring-boot-starter-tomcat\2.5.5\spring-boot-starter-tomcat-2.5.5.jar;D:\workspace\mavenLocalRepository\org\apache\tomcat\embed\tomcat-embed-core\9.0.53\tomcat-embed-core-9.0.53.jar;D:\workspace\mavenLocalRepository\org\apache\tomcat\embed\tomcat-embed-el\9.0.53\tomcat-embed-el-9.0.53.jar;D:\workspace\mavenLocalRepository\org\apache\tomcat\embed\tomcat-embed-websocket\9.0.53\tomcat-embed-websocket-9.0.53.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-web\5.3.10\spring-web-5.3.10.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-beans\5.3.10\spring-beans-5.3.10.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-webmvc\5.3.10\spring-webmvc-5.3.10.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-aop\5.3.10\spring-aop-5.3.10.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-context\5.3.10\spring-context-5.3.10.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-expression\5.3.10\spring-expression-5.3.10.jar;D:\workspace\mavenLocalRepository\mysql\mysql-connector-java\8.0.26\mysql-connector-java-8.0.26.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-core\5.3.10\spring-core-5.3.10.jar;D:\workspace\mavenLocalRepository\org\springframework\spring-jcl\5.3.10\spring-jcl-5.3.10.jar;H:\ProframFiles\IntelliJ IDEA 2019.3\lib\idea_rt.jar" com.github.misterchangray.user.FasterUserApplication
Java HotSpot(TM) 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
Connected to the target VM, address: '127.0.0.1:0', transport: 'socket'
Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended.   ____          _            __ _ _/\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \\\/  ___)| |_)| | | | | || (_| |  ) ) ) )'  |____| .__|_| |_|_| |_\__, | / / / /=========|_|==============|___/=/_/_/_/:: Spring Boot ::                (v2.5.5){"instant":{"epochSecond":1693362240,"nanoOfSecond":139475300},"thread":"main","level":"INFO","loggerName":"com.github.misterchangray.user.FasterUserApplication","message":"Starting FasterUserApplication using Java 13.0.1 on DESKTOP-KVEV94V with PID 21844 (D:\\workspace\\playground\\faster-demo\\faster-user\\target\\classes started by Miste in D:\\workspace\\playground\\faster-demo)","endOfBatch":false,"loggerFqcn":"org.apache.commons.logging.LogAdapter$Log4jLog","contextMap":{},"threadId":1,"threadPriority":5,"project":"demo1"}
{"instant":{"epochSecond":1693362240,"nanoOfSecond":202819200},"thread":"main","level":"DEBUG","loggerName":"com.github.misterchangray.user.FasterUserApplication","message":"Running with Spring Boot v2.5.5, Spring v5.3.10","endOfBatch":false,"loggerFqcn":"org.apache.commons.logging.LogAdapter$Log4jLog","contextMap":{},"threadId":1,"threadPriority":5,"project":"demo1"}
{"instant":{"epochSecond":1693362240,"nanoOfSecond":203813500},"thread":"main","level":"INFO","loggerName":"com.github.misterchangray.user.FasterUserApplication","message":"No active profile set, falling back to default profiles: default","endOfBatch":false,"loggerFqcn":"org.apache.commons.logging.LogAdapter$Log4jLog","contextMap":{},"threadId":1,"threadPriority":5,"project":"demo1"}
{"instant":{"epochSecond":1693362240,"nanoOfSecond":734127100},"thread":"main","level":"INFO","loggerName":"org.springframework.boot.web.embedded.tomcat.TomcatWebServer","message":"Tomcat initialized with port(s): 8089 (http)","endOfBatch":false,"loggerFqcn":"org.apache.commons.logging.LogAdapter$Log4jLog","contextMap":{},"threadId":1,"threadPriority":5,"project":"demo1"}
{"instant":{"epochSecond":1693362240,"nanoOfSecond":745097900},"thread":"main","level":"INFO","loggerName":"org.apache.coyote.http11.Http11NioProtocol","message":"Initializing ProtocolHandler [\"http-nio-8089\"]","endOfBatch":false,"loggerFqcn":"java.util.logging.Logger","contextMap":{},"threadId":1,"threadPriority":5,"project":"demo1"}
http://www.lryc.cn/news/151562.html

相关文章：

  • 某次护网红队getshell的经历
  • C#实现日期选择器、显示当地时间、跑马灯等功能
  • 如何让看书变听书？
  • pytorch异常——loss异常，不断增大，并且loss出现inf
  • Lua学习（一）
  • Python：列表推导式
  • 应急三维电子沙盘数字孪生系统
  • LeetCode每日一题：1654. 到家的最少跳跃次数（2023.8.30 C++）
  • 数据结构例题代码及其讲解-栈与队列
  • 【Spark】Pyspark RDD
  • 数学建模：Logistic回归预测
  • 一个面向MCU的小型前后台系统
  • 软件外包开发人员分类
  • HTML 元素被定义为块级元素或内联元素
  • 单调递增的数字【贪心算法】
  • gnuradio-hackrf_info.exe -FM频率使用
  • JVM学习(三)--生产环境的线程问题诊断
  • PHP数组处理$arr1转换为$arr2
  • ATF(TF-A)安全通告 TFV-10 (CVE-2022-47630)
  • 详解 SpringMVC 中获取请求参数
  • Message: ‘chromedriver‘ executable may have wrong permissions.
  • 每日一题 1372二叉树中的最长交错路径
  • 【力扣每日一题】2023.9.2 最多可以摧毁的敌人城堡数量
  • kotlin实现java的单例模式
  • 使用 KeyValueDiffers 检测Angular 对象的变化
  • Macos 10.13.2安装eclipse
  • Android逆向学习（一）vscode进行android逆向修改并重新打包
  • 【深入浅出设计模式--状态模式】
  • Debezium系列之：Debezium Server在生产环境大规模应用详细的技术方案
  • Echart笔记