Score:0

Syslog-ng enable TCP framing

to flag

I'm trying to send file entries as messages via TCP, where syslog-ng is in a container and it is sending to another container. I've had two different attempts both with problematic behavior. The first configuration:

@version: 3.31
source s_file {
    file("/var/log/my_file.json" follow_freq(1) flags(no-parse));
};

template log_template {
    template("MSGSTART${MESSAGE}MSGEND");
};

class SngResolver(object):
    def init(self, options):
        """
        Initializes the parser
        """
        self.counter = 0
        return True
    def parse(self, log_message):
        log_message["SYSUPTIME"] = subprocess.check_output(['cat', '/proc/uptime']).decode('utf-8')
        log_message["SEQUENCEID"] = str(self.counter)
        self.counter += 1
        # return True, other way message is dropped
        return True
};

parser p_resolver {
    python(
        class("SngResolver")
    );
};


# Define the destination for Suricata logs
destination d_container {
    syslog("my_other_container" transport("tcp") port(1234) template(log_template));    
};


# Define the log path for Suricata logs
log {
    source(s_file);
    parser(p_resolver);

    destination(d_container);
};

In this method, sometimes the message received started with the number of bytes in the coming message, say 400. other times, they did not and went straight to the message.

Later, I changed the destination to use network instead of syslog. Now, there is no framing.

I don't mind if I have to use TCP, UDP, whatever. I have a golang received connected to a TCP socket and I want it to read, one message at a time, and parse it. How is this achievable? Thanks

bazsi77 avatar
ke flag
I would be very surprised if the syslog() driver would not send framing in the case above, it has been in production for a long time. Can you perhaps capture the traffic on the wire as syslog-ng sends it out? or as your go program receives it? A quick check of the related code looks sane to me, even though I haven't looked at it for a while. If the receiving side gets out of sync, it might be possible that you don't find the octet count where it is expected to be found.
Score:0
ke flag

The syslog() destination would use the octet-counted framing format on transport(tcp) and transport(tls) as described in RFC5425. It will NOT use framing on transport(udp) as in that case the datagram delianates messages using the packet boundaries. The UDP transport is described in RFC5426.

I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.