Join Log in
Score:0
AnujaDx avatar Server

Why this java processes are not ending once cronjob has executed in the AIX server?

vn flag
AnujaDx

I have set a crontab for execute a java service in every 8 hours in AIX(7.1) operating system. Once the crontab has done the job, the JAVA service should be end. This cronjob is working properly. But java service has not been closing its session at the end. Also this java service takes 3-4 hours to process the files.

Java sessions are accumulating day by day and eventually it leads to idle the application server.

Java Version 1.8
AIX 7.1

Crontab entry:

00 8 * * * cd /ibl/qnb/uat/app/adm/ADM_AUTOMATE_SERV && ./RUN.sh > /ibl/qnb/uat/logs/adm/ADM_AUTOMATE_SERV/run.log

If I execute the above statement manually, cronjob runs properly and closing the session at the end.

RUN.sh file:

/usr/java8_64/bin/java -Xmx1024m -jar ADM_AUTOMATE_SERV.jar

Below are the java sessions still running in the application server. Please have a look and advise what I'm going wrong?

30932996  7340260   0   Mar 25      -  0:00 sh -c cd /ibl/qnb/uat/app/adm/ADM_AUTOMATE_SERV && ./RUN.sh > /ibl/qnb/uat/logs/adm/ADM_AUTOMATE_SERV/run.log
31129600 29425670  21   Mar 30      - 2953:54 /usr/java8_64/bin/java -Xmx1024m -jar ADM_AUTOMATE_SERV.jar
32505878 33620190  25 08:00:00      - 216:49 /usr/java8_64/bin/java -Xmx1024m -jar ADM_AUTOMATE_SERV.jar
32833638 40304674  25   Mar 28      - 4183:00 /usr/java8_64/bin/java -Xmx1024m -jar ADM_AUTOMATE_SERV.jar
33620190  7340260   0 08:00:00      -  0:00 sh -c cd /ibl/qnb/uat/app/adm/ADM_AUTOMATE_SERV && ./RUN.sh > /ibl/qnb/uat/logs/adm/ADM_AUTOMATE_SERV/run.log
33947736  7340260   0   Apr 04      -  0:00 sh -c cd /ibl/qnb/uat/app/adm/ADM_AUTOMATE_SERV && ./RUN.sh > /ibl/qnb/uat/logs/adm/ADM_AUTOMATE_SERV/run.log
34406468 35389628  22   Apr 03      - 965:58 /usr/java8_64/bin/java -Xmx1024m -jar ADM_AUTOMATE_SERV.jar
34471988  7340260   0   Mar 31      -  0:00 sh -c cd /ibl/qnb/uat/app/adm/ADM_AUTOMATE_SERV && ./RUN.sh > /ibl/qnb/uat/logs/adm/ADM_AUTOMATE_SERV/run.log
35389628  7340260   0   Apr 03      -  0:00 sh -c cd /ibl/qnb/uat/app/adm/ADM_AUTOMATE_SERV && ./RUN.sh > /ibl/qnb/uat/logs/adm/ADM_AUTOMATE_SERV/run.log
37290036 34471988  24   Mar 31      - 2444:53 /usr/java8_64/bin/java -Xmx1024m -jar ADM_AUTOMATE_SERV.jar
38863052  7340260   0   Apr 02      -  0:00 sh -c cd /ibl/qnb/uat/app/adm/ADM_AUTOMATE_SERV && ./RUN.sh > /ibl/qnb/uat/logs/adm/ADM_AUTOMATE_SERV/run.log
39452824 30212288  30   Mar 26      - 5463:39 /usr/java8_64/bin/java -Xmx1024m -jar ADM_AUTOMATE_SERV.jar
40304674  7340260   0   Mar 28      -  0:00 sh -c cd /ibl/qnb/uat/app/adm/ADM_AUTOMATE_SERV && ./RUN.sh > /ibl/qnb/uat/logs/adm/ADM_AUTOMATE_SERV/run.log
41484458  7340260   0   Apr 01      -  0:00 sh -c cd /ibl/qnb/uat/app/adm/ADM_AUTOMATE_SERV && ./RUN.sh > /ibl/qnb/uat/logs/adm/ADM_AUTOMATE_SERV/run.log
43188280 23003326  27   Mar 22      - 8316:22 /usr/java8_64/bin/java -Xmx1024m -jar ADM_AUTOMATE_SERV.jar
46596146 20512790  29   Mar 27      - 4821:18 /usr/java8_64/bin/java -Xmx1024m -jar ADM_AUTOMATE_SERV.jar
46792934 27132138  26   Mar 24      - 6883:58 /usr/java8_64/bin/java -Xmx1024m -jar ADM_AUTOMATE_SERV.jar
46923976 30932996  26   Mar 25      - 6177:01 /usr/java8_64/bin/java -Xmx1024m -jar ADM_AUTOMATE_SERV.jar

Highly appreciate your time and efforts for this matter.

44
0 + 0
aix
fo flag
glenn jackman
Also asked at https://stackoverflow.com/q/71757596/7552
0
Reply
AnujaDx avatar
vn flag
AnujaDx
@glennjackman Yes, but meanwhile posted here since this is linux specific portal.
0
Reply
ae flag
Juraj Martinka
Do you have logs that you can share? What happens if you capture a thread dump of the pending java processes - where they are stuck? You can take a thread dump via `jcmd` or simple `kill -3 <java process pid>` (in this case, it will be printed on stdout)
0
Reply
Jeff Schaller avatar
nf flag
Jeff Schaller
I wonder if ADM_AUTOMATE_SERV is relying on an environment variable that's present in your interactive shell but not in a non-interactive shell. Try comparing the output of `env` from your interactive shell against running it from `RUN.sh`.
0
Reply
fo flag
glenn jackman
@AnujaDx, it can be frustrating as someone who provides the time and effort to answer in one site only to later find the same question at a different site with more comments and better answers. On some StackExchange sites, cross-posting is grounds to close the question. At least you could link the two questions yourself.
1
Reply
Score:2
avatar Server
ae flag
Juraj Martinka

I first made a comment but I think writing a proper answer will be more meaningful.

The first thing I would check is logs: is there anything meaningful in run.log? (and perhaps you could append to that file instead of overwriting it every time the job runs)

Then check the user/group under which the process is running - is it different when it runs from cron than when you run it yourself?

A very useful technique for figuring out where a Java/JVM app is stuck is making a thread dump - there are several ways do it:

  • my favorite tool is jcmd - you can simply do

    jcmd <pid> Thread.print > thread.$(date "+%F_%H-%M-%S").dump

  • another quick way is to send SIGQUIT signal: kill -3 <pid>

    • in this case, the thread dump will be printed to the stdout of the process

Having a thread dump, you can examine the application threads and see where they are stuck.

You can also try strace to capture system calls made by the app - that can tell you what it's doing too.

0 + 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.