Wednesday, February 09, 2011

Strange Nagios Error Solved

This morning I added some new services to a server, but they wouldn't move out of "Pending" status. The error I received was
"Feb 9 09:27:36 nagios: Warning: Check result queue contained results for service '' on host '', but the service could not be found! Perhaps you forgot to define the service in your config files?"

I stopped the nagios service and ran ps -ef | grep nagios. To my surprise there was still a Nagios instance running. This means two things. First the init script that comes with Nagios is borked and doesn't correctly check for running nagios instances. Second, I somehow started a second Nagios instance.

I thought something was up in the first place because every other refresh or so of the Nagios web view I would either see the three pending services or I wouldn't. This was my first clue that something was borked. I then went to tail /var/log/messages and saw the error message. Then I started investigating the issue with the help of Google. Once I saw that there where two instances of Nagios things started to make sense.

I killed the second Nagios instance and any children processes (in my case ndo2db) and then restarted nagios via the init script.

Once I had everything up and running (but only one instance) I was able to successfully check my new services.

6 comments:

Anonymous said...

I had the same issue. It got resolved after killing that extra process. Thank you so much for your post.

Anonymous said...

I got the same problem! thanks buddy!

Kuldeep Sharma said...

Thanks Buddy :)

Scott said...

Thank you! I had the same problem and killing the extra process resolved it.

Anonymous said...

Thanks for that!!

kamadelva said...

2 years after your post, the same strange behavior still around. Your advice worked like a charm !