Join Log in
Score:0
avatar Server

Discard less important logs during overload with fluentbit or fluentd

in flag
Gregor

Assuming I could identify more or less important logs by pattern-matching them, is there a way to configure fluentd (or fluentbit) to do intelligent shedding (discards) when it starts to buffer to much (back-pressure from output)?

Are there other log-processing filters that would do this?

Basically, under low/normal loads I want to pass all the logs, but during overload or spike situation I would like to sacrifice some less important logs in order to preserve the more important logs.

A periodic summary log of discarded count would also be useful but not a strict requirement.

Edit: re-ordering of logs could be a problem, so I would like to find a solution that does not do that.

58
0 + 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.