[Zenoss-dev] watchdog problems after upgrade from 2.1.2 to 2.1.7.0

[jonny]

We just upgraded our core 2.1.2 to 2.1.7.0 (2.2), and now the majority of the  Daemons are having issues communicating with watchdog. We checked zensocket, the sticky bit is set, and our zenoss and root user has access to it.  The system is for the most part non-functional at this point since the upgrade.  Here is a sample from the logs of the Daemons having the issue.   It looks like an issue with the Daemons being able to communicate with watchdog. Can anyone help out of have suggestions?

zenhub:

2008-01-15 11:45:58 ERROR watchdog: Restarting zenhub
2008-01-15 11:47:58 ERROR watchdog: Timeout: getting initial connection from process
2008-01-15 11:50:07 ERROR watchdog: Restarting zenhub
2008-01-15 11:52:07 ERROR watchdog: Timeout: getting initial connection from process


zenstatus log:

2008-01-15 11:48:15 INFO zen.ZenStatus: delete pidfile /opt/zenoss/var/zenstatus.py.pid
2008-01-15 11:48:15 INFO zen.ZenStatus: Daemon ZenStatus shutting down
2008-01-15 11:48:15 INFO zen.ZenStatus: Daemon ZenStatus shutting down
2008-01-15 11:48:15 INFO zen.ZenStatus: zenstatus shutting down
2008-01-15 11:48:15 ERROR watchdog: Timeout: getting initial connection from process


zenperfsnmp log: 

2008-01-15 11:48:34 INFO zen.zenperfsnmp: delete pidfile /opt/zenoss/var/zenperfsnmp.py.pid
2008-01-15 11:48:34 INFO zen.zenperfsnmp: Daemon zenperfsnmp shutting down
2008-01-15 11:48:34 INFO zen.zenperfsnmp: Daemon zenperfsnmp shutting down
2008-01-15 11:48:34 INFO zen.zenperfsnmp: zenperfsnmp shutting down
2008-01-15 11:48:35 ERROR watchdog: Timeout: getting initial connection from process
2008-01-15 11:50:44 ERROR watchdog: Restarting zenperfsnmp



zencommand log: 

008-01-15 11:48:42 INFO zen.zencommand: delete pidfile /opt/zenoss/var/zencommand.py.pid
2008-01-15 11:48:42 INFO zen.zencommand: Daemon zencommand shutting down
2008-01-15 11:48:42 INFO zen.zencommand: Daemon zencommand shutting down
2008-01-15 11:48:42 INFO zen.zencommand: zencommand shutting down
2008-01-15 11:48:43 ERROR watchdog: Timeout: getting initial connection from process
2008-01-15 11:50:52 ERROR watchdog: Restarting zencommand


zenprocess log:

2008-01-15 11:48:47 INFO zen.zenprocess: delete pidfile /opt/zenoss/var/zenprocess.py.pid
2008-01-15 11:48:47 INFO zen.zenprocess: Daemon zenprocess shutting down
2008-01-15 11:48:47 INFO zen.zenprocess: Daemon zenprocess shutting down
2008-01-15 11:48:47 INFO zen.zenprocess: zenprocess shutting down
2008-01-15 11:48:47 ERROR watchdog: Timeout: getting initial connection from process


zenwin log: 

008-01-15 11:48:50 INFO zen.zenwin: delete pidfile /opt/zenoss/var/zenwin.py.pid
2008-01-15 11:48:50 INFO zen.zenwin: Daemon zenwin shutting down
2008-01-15 11:48:50 INFO zen.zenwin: Daemon zenwin shutting down
2008-01-15 11:48:50 INFO zen.zenwin: zenwin shutting down
2008-01-15 11:48:51 ERROR watchdog: Timeout: getting initial connection from process



zeneventlog log:

2008-01-15 11:48:56 INFO zen.zeneventlog: delete pidfile /opt/zenoss/var/zeneventlog.py.pid
2008-01-15 11:48:56 INFO zen.zeneventlog: Daemon zeneventlog shutting down
2008-01-15 11:48:56 INFO zen.zeneventlog: Daemon zeneventlog shutting down
2008-01-15 11:48:56 INFO zen.zeneventlog: zeneventlog shutting down
2008-01-15 11:48:57 ERROR watchdog: Timeout: getting initial connection from process


zenwinmodeler log: 

2008-01-15 11:49:00 INFO zen.zenwinmodeler: delete pidfile /opt/zenoss/var/zenwinmodeler.py.pid
2008-01-15 11:49:00 INFO zen.zenwinmodeler: Daemon zenwinmodeler shutting down
2008-01-15 11:49:00 INFO zen.zenwinmodeler: Daemon zenwinmodeler shutting down
2008-01-15 11:49:00 INFO zen.zenwinmodeler: zenwinmodeler shutting down
2008-01-15 11:49:01 ERROR watchdog: Timeout: getting initial connection from process




-------------------- m2f --------------------

Read this topic online here:
http://community.zenoss.com/forums/viewtopic.php?p=15338#15338

-------------------- m2f --------------------