Score:0

Finding source of high IOWait

cn flag

I have a 5 node elasticsearch cluster. One host has had consistently high IOWait% (40+) for several weeks. The others seem fine (<10%).

Notable:

  • Host in question is not the cluster master
  • New indexes are randomly distributed among all 5 hosts
  • IOTop shows that all high-wait-% processes are elasticsearch (IE no bots, viruses, etc)
  • All data is stored on a SAN with a 10G bonded connection
  • All hosts are configured the same

Running:

  • CentOs 7.x
  • 128G RAM

Suggestions on other tools to use to ferret out the problem(s)?

shodanshok avatar
ca flag
Can you show the output of `iotop -a -n 5` and `iostat -x -k 1 5`?
cn flag
Both provide a slew of data. You want it all?
shodanshok avatar
ca flag
Yes, I need them to try to understand. You can use something as pastebin to show the output.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.