Java | MyBlog

在金融级手机银行中，日志是系统的黑匣子：工程团队用它复盘链路，审计与监管用它证明事实。单体年代，落盘归档就够用；微服务与容器云时代，实例倍增、日志暴涨；高并发下，过度日志会吃满 I/O、拖垮 CPU，主链路也会跟着失速。因此，可用的日志系统必须同时做到高性能、高可用、低成本，并满足合规。我们把方法归纳为四类：采集、配置、规范、压测。一、采集——应用只写 stdout，平台全权接管。 LogAgent: 在节点侧采集标准输出，送到Collector，再分发到 Kafka/ES/对象存储；容器重启不丢、查询口径一致。大对象不上链：交易日志只写摘要（size、hash、关键字段）；全量请求/响应、慢SQL走异步旁路，主链路不被堵。链路可降级：多副本冗余；背压时先丢DEBUG/INFO，保WARN/ERROR。成本可控：统计 GB/天、监控backlog与延迟；在日志中注入traceId/spanId，确保跨服务可追踪。协议选择: 以gRPC/HTTP + 批量压缩为主，极端吞吐再评估Netty/TCP的复杂度。二、配置——把“写日志”的成本从主线程挪走。使用AsyncAppender，主线程入队、后台批量写；注：队列大小估算：queueSize ≈ QPS × 单条日志字节数 × 可容忍积压秒数（受 JVM 可用内存约束）。设置条件丢弃，只丢低级别。格式输出用JSON，时间戳写epochMillis，展示端再格式化。统一字段：traceId、spanId、app、pod、instance、thread、uid、ip、channel、uri、method、status、rt_ms；通过 MDC 注入模板输出。归档按时间 + 大小 + 级别组合；ERROR长保，INFO/DEBUG短保； Agent 侧设置每容器限速，防止I/O被日志占满。代码示例 2.1 logback.xml（异步、JSON、分级归档、独立 payload/ERROR 通道）放置：src/main/resources/logback.xml 说明： APP_FILE：INFO/DEBUG 主通道（短保，JSON，Size+Time 归档）。 ERROR_FILE：ERROR 独立通道（长保）。 PAYLOAD_FILE：大负载/慢 SQL 旁路通道（短保+隔离）。三者均用 AsyncAppender，业务线程永不阻塞。 <configuration>  <property name="APP" value="mobile-bank"/> <property name="LOG_HOME" value="/data/logs/${APP}"/>  <appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${LOG_HOME}/app.log</file> <encoder class="net.logstash.logback.encoder.LogstashEncoder"> <includeMdc>true</includeMdc> <timeZone>UTC</timeZone> <timestampPattern>UNIX_MILLIS</timestampPattern>  <fieldNames> <timestamp>ts</timestamp> <level>lvl</level> <thread>th</thread> <logger>logger</logger> <message>msg</message> </fieldNames> </encoder> <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy"> <fileNamePattern>${LOG_HOME}/app.%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern> <maxFileSize>128MB</maxFileSize>  <maxHistory>14</maxHistory>  <totalSizeCap>50GB</totalSizeCap>  </rollingPolicy>  <filter class="ch.qos.logback.classic.filter.LevelFilter"> <level>ERROR</level> <onMatch>DENY</onMatch> <onMismatch>NEUTRAL</onMismatch> </filter> </appender>  <appender name="ERROR_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${LOG_HOME}/error.log</file> <encoder class="net.logstash.logback.encoder.LogstashEncoder"> <includeMdc>true</includeMdc> <timeZone>UTC</timeZone> <timestampPattern>UNIX_MILLIS</timestampPattern> </encoder> <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy"> <fileNamePattern>${LOG_HOME}/error.%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern> <maxFileSize>64MB</maxFileSize> <maxHistory>90</maxHistory>  <totalSizeCap>100GB</totalSizeCap> </rollingPolicy>  <filter class="ch.qos.logback.classic.filter.LevelFilter"> <level>ERROR</level> <onMatch>ACCEPT</onMatch> <onMismatch>DENY</onMismatch> </filter> </appender>  <appender name="PAYLOAD_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${LOG_HOME}/payload.log</file> <encoder class="net.logstash.logback.encoder.LogstashEncoder"/> <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy"> <fileNamePattern>${LOG_HOME}/payload.%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern> <maxFileSize>64MB</maxFileSize> <maxHistory>3</maxHistory>  </rollingPolicy> </appender>   <appender name="ASYNC_APP" class="ch.qos.logback.classic.AsyncAppender"> <appender-ref ref="APP_FILE"/> <queueSize>20000</queueSize>  <discardingThreshold>30</discardingThreshold>  <neverBlock>true</neverBlock>  <includeCallerData>false</includeCallerData> </appender> <appender name="ASYNC_ERROR" class="ch.qos.logback.classic.AsyncAppender"> <appender-ref ref="ERROR_FILE"/> <queueSize>5000</queueSize>  <discardingThreshold>0</discardingThreshold> <neverBlock>true</neverBlock> </appender> <appender name="ASYNC_PAYLOAD" class="ch.qos.logback.classic.AsyncAppender"> <appender-ref ref="PAYLOAD_FILE"/> <queueSize>5000</queueSize> <discardingThreshold>10</discardingThreshold>  <neverBlock>true</neverBlock> </appender>  <root level="INFO"> <appender-ref ref="ASYNC_APP"/> <appender-ref ref="ASYNC_ERROR"/> </root>  <logger name="payload.logger" level="INFO" additivity="false"> <appender-ref ref="ASYNC_PAYLOAD"/> </logger> </configuration> Maven 依赖（使用 JSON 编码器）： <dependency> <groupId>net.logstash.logback</groupId> <artifactId>logstash-logback-encoder</artifactId> <version>7.4</version> </dependency> 2.2 MDC 过滤器（统一注入字段）放置：src/main/java/com/example/logging/MdcFilter.java 作用：在每个请求进入时，把统一字段放进 MDC，日志模板自动带出；请求结束后必须清理。 package com.example.logging; import org.slf4j.MDC; import javax.servlet.*; import javax.servlet.http.HttpServletRequest; import java.io.IOException; import java.net.InetAddress; public class MdcFilter implements Filter { @Override public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException { HttpServletRequest request = (HttpServletRequest) req; try { // 1) traceId / spanId：来自网关/APM（无则本地生成保底） String traceId = request.getHeader("X-Trace-Id"); if (traceId == null || traceId.isEmpty()) { traceId = java.util.UUID.randomUUID().toString(); } MDC.put("traceId", traceId); String spanId = request.getHeader("X-Span-Id"); if (spanId != null) MDC.put("spanId", spanId); // 2) 统一基础维度 MDC.put("app", System.getenv().getOrDefault("APP_NAME", "mobile-bank")); MDC.put("pod", System.getenv().getOrDefault("HOSTNAME", "N/A")); MDC.put("instance", InetAddress.getLocalHost().getHostName()); MDC.put("thread", Thread.currentThread().getName()); // 3) 请求维度 MDC.put("uri", request.getRequestURI()); MDC.put("method", request.getMethod()); MDC.put("ip", request.getRemoteAddr()); MDC.put("channel", request.getHeader("X-Channel") == null ? "unknown" : request.getHeader("X-Channel")); // 4) 业务维度（示例：用户ID） String uid = request.getHeader("X-User-Id"); if (uid != null) MDC.put("uid", uid); chain.doFilter(req, res); } finally { // 防止线程复用导致“脏 MDC” MDC.clear(); } } } 注册 Filter（Spring Boot）： package com.example.logging; import org.springframework.boot.web.servlet.FilterRegistrationBean; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @Configuration public class FilterConfig { @Bean public FilterRegistrationBean<MdcFilter> mdcFilterRegistration() { FilterRegistrationBean<MdcFilter> reg = new FilterRegistrationBean<>(); reg.setFilter(new MdcFilter()); reg.addUrlPatterns("/*"); reg.setOrder(1); // 尽量靠前，保证后续日志都有 MDC return reg; } } 2.3 Agent 侧限速（Fluent Bit → Kafka 示例） “每容器限速”通常结合匹配规则 + 背压/节流插件 + 后端限额实现。以下为简化示例。 ...