Score:0

Server

Graylog - data from newest indice not returned on search

OpenHaus

9/9/23, 9:07 PM

I have a Graylog server (newest version) collecting data from nginx. It´s been running for a while. I retrieve the collected data by querying my Elasticsearch cluster (v7 newest version) which consists of 4 nodes ATM. This all work(s/ed) fine. Now I have the problem that Elasticsearch won´t return the newest data. I checked that it exists. I do not see any differences to previous data stored. I did have a downtime (aka crash) because my disks where running full. All statuses show that the system is running fine again.

I checked everything I could find but ran out of ideas. Where do I need to look to solve this? I appreciate any ideas!

Update

What I also noticed is that I increased the Index Replicas from 0 (default) to 1 for the default index set but there seems to be no replicas. Shouldn´t those be created automatically after changing the settings?

0 + 0

elasticsearch

graylog

Swisstone

9/12/23, 8:53 PM

Can you take a look at the Nodes status page and see if the nodes aren't storing data in the Disk Journal?

OpenHaus

9/12/23, 9:45 PM

Thank you for your comment @Swisstone If you are referring to the Graylog Disk Journal then yes there is data written to it.

Score:1

Server

cr3a7ure

9/22/23, 3:21 PM

I am not sure exactly how to guide you, but have you looked at the log agent?

From my experience, I can not think of a simple reason that ES would not return data that are digested, except when querying with wrong filters such as wrong datetime (timezone changes).

After a crash on the ElasticSearch side of things, some times filebeat requires a restart. You should check its logs too, it might help you pin point the problem.

On the replication option, now.

Τhe option index.number_of_replicas is a dynamic option and should be updated live. If this option is passed through the index template it gets applied on index rotation ( aka, a new index is spawned ). Otherwise you might need to update it on the index itself

0 + 0

OpenHaus

9/23/23, 9:14 PM

Thank you @cr3a7ure ! I rotated the active write index manually which created the replicas. I do not think filebeat is used in our setup. Did I read correctly that query logging is only available with a paid subscriptions? The logs do not display any helpful information.

cr3a7ure

9/24/23, 7:56 AM

I brought up filebeat as a typical example agent. I have no hands on experience with Graylog. I can see that they use an agent called [Forwarder](https://docs.graylog.org/docs/forwarder). Other than that, I would ask in their forums too, about subscriptions and pricing/features.

OpenHaus

9/24/23, 10:37 AM

I solved it. At the end it wasn´t Elastic at all. You were right there was a timezone offset which let to the missing results.

cr3a7ure

9/24/23, 11:13 AM

Happy to help. I had stepped on it by myself once and learned the hard way!

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Graylog - data from newest indice not returned on search

TH: Graylog - ข้อมูลจากดัชนีล่าสุดไม่ส่งคืนในการค้นหา

RO: Graylog - datele din cel mai nou indice nu au fost returnate la căutare

RU: Graylog - данные из новейшего индекса не возвращаются при поиске

VI: Graylog - dữ liệu từ chỉ mục mới nhất không được trả về khi tìm kiếm

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.