ELK 日志处理
评论(0) 浏览量(99)

Loki is another logging stack that seems to be lighter weighted than the ELK stack,

没有docker用,

elk / Loki 

 using sleuth and zippkin

These 3 are super heavy, the compressed archives are 500M big, if extracted, they take quite a lot of time, and path string can't exceed 255 such error happens, they take up many GB spaces when extracting, that's so horrible,

using up to 5 hours to install , and 128GB ssd, I am afraid I have to give up this kibana installation, 

The elk docker images

image.png

image.png


docker-compose -f composefile.yaml up, for windows, it will throw "too many colons", also use /d/pathofyaml/config.yaml, this works for docker compose.


So,grafana can display graphs, loki collects logs, prometheus collects service info such as zookeeper nodes,

这里为springcloud添加上prometheus的 servicename 的job, 在prometheus的配置中添加上单个服务的地址,需要慢慢看一下文档, 直接用docker compose组合的弊端就是一下子量太大,其实每一个都好多东西;

这里切换一下用gradle 8.10看看


refs:


https://auth0.com/blog/spring-boot-logs-aggregation-and-monitoring-using-elk-stack/


https://developer.okta.com/blog/2021/07/26/spring-cloud-sleuth#:~:text=Spring%20Cloud%20Sleuth%20allows%20you%20to%20aggregate%20and,just%20filter%20the%20log%20by%20the%20thread%20ID.


https://phoenixnap.com/kb/elk-stack-docker


https://dev.to/luafanti/spring-boot-logging-with-loki-promtail-and-grafana-loki-stack-aep


https://juejin.cn/post/7303741345322745908

https://grafana.com/docs/loki/latest/

https://prometheus.io/docs/prometheus/latest/installation/


https://statuslist.app/uptime-monitoring/zookeeper/self-hosted/how-to-setup-zookeeper-monitoring-prometheus-grafana/


https://piotrminkowski.com/2023/07/05/logging-in-spring-boot-with-loki/


https://grafana.com/docs/loki/latest/send-data/promtail/installation/


https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Super_POM



loki 遇到问题了, 本机都启动后, springboot 怎么也起不了, 好久后发生错误:

image.png

,loki 用的docker 配置的最新版本,不知道远程怎么样,反正本机启动不了。

pooling 轮询也是做一个时间间隔更新。

然后找到loki的官方配置页:

https://loki4j.github.io/loki-logback-appender/docs/configuration

设置 logback appender的超时时间和重试次数和单批次上传大小:

image.png

再次启动,很快结果就弹出不能上传


image.png

, docker 启动的loki, 不能用呢, logback 的级别设置为DEBUG,


image.png

springboot debug启动日志很大, 很多/, 这里报个检查发送队列容量,再尝试设置一下,

如果把loki的docker 停掉,马上就会出现不能连接错误,所以实际这里上是连接上loki的docker的了,

但是传不上去,

重新docker run loki:3.0.0, 因为不知道那个latest是什么版本,出现badrequest, 说size 过大, 所以这里需要调整一下loki单批次上传item的个数,

然后为logback添加多个appender, 并且设置 DEBUG同时输出到多个appender

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>
    <appender name="LOKI" class="com.github.loki4j.logback.Loki4jAppender">
        <batchTimeoutMs>2000</batchTimeoutMs>
        <maxRetries>1</maxRetries>
        <sendQueueMaxBytes>419430400</sendQueueMaxBytes>
        <batchMaxItems>2</batchMaxItems>
        <http class="com.github.loki4j.logback.ApacheHttpSender">
            <url>http://192.168.2.155:3200/loki/api/v1/push</url>
            <connectionTimeoutMs>2000</connectionTimeoutMs>
            <requestTimeoutMs>3000</requestTimeoutMs>
            <connectionKeepAliveMs>5000</connectionKeepAliveMs>
        </http>
        <format>
            <label>
                <pattern>app=my-app,host=${HOSTNAME},level=%level</pattern>
            </label>
            <message>
                <pattern>l=%level h=${HOSTNAME} c=%logger{20} t=%thread | %msg %ex</pattern>
            </message>
        </format>
    </appender>

    <root level="DEBUG">
        <appender-ref ref="STDOUT" />
        <appender-ref ref="LOKI" />
    </root>
</configuration>

,重新用docker拉loki3.0跑3200端口,然后看到debug里面也输出信息:

image.png

,loki的post地址没有返回结果,然后就无限在转,这个不知道loki的推送还需要什么,这个挺麻烦,

查看docker中loki的配置,auth_enabled设置为fase,

image.png


然后去查loki http api doc:

https://grafana.com/docs/loki/latest/reference/loki-http-api/

本地出现 751000个消息 dropped,说明消息没有发成功,不知道版本还是什么,这个有enterprise和开源版本的grafana, loki不知道是什么了,反正发不成功,连接是成功的,所以这个东西,可能就不能用了么,不过这个调试日志的过程和不停找loki各种配置的时间的设置对于这种http的包,还是值得记录的,

然后,对于开源的东西,接下来就是,build from source了, 

可能要自己build一下源代码,才能确保能不能用和怎么用了,

然后jdk8是一个特殊的存在,要去这个lokii4j这个包的地方查一上兼容性问题,使用对应版本的依赖,

其实对于单体来讲,可能自己写一个log appender,也是挺好的,然后日志上传的时候,设置一下本地批处理队列大小,到达某个大小就将队列里面堆积的log发送到服务器,删除队列里面的消息,



using the ELK stack with docker compose:

https://dzone.com/articles/deploying-springboot-in-ecs-part-1


没有登录不能评论