Re: [cobalt-users] Newbie to list, problem with Qube3 desktop andadmin interface

[David Shugarts <Azimuth@xxxxxxxxxxxxxx>]

> 
> David Shugarts wrote:
> 
>> Thanks for any help you can provide. I have rebooted the entire machine
>> several times, but the admin server will not come back. I am not sure of the
>> specific Telnet command for restarting it individually (I thought "start
>> httpd" from root worked for both).
> 
> Restarting the entire Raq should have restarted all the services.  I
> generall restart services this way:
> 
> # /etc/rc.d/init.d/<servicename> restart
> 
> where <servicename> is the name of the service I'm restarting.
> 
> To see a list of the services that'll restart this way, do:
> 
> # ls -l /etc/rc.d/init.d/*
> 
>> However, it looks as though I have other problems that were not evident
>> until a fellow Listee suggested that I look at the error logs. We
>> corresponded a bit off-list and he feels I am in deep trouble. So I have
>> been making plans to restore the server, and I have obtained the OS 6.4 disk
>> image from Cobalt to do that job. I have just crossed my fingers that the
>> essential processes of the Cobalt will carry on until I can do it.
> 
> Reading between the lines it sounds as if you're saying you were hacked;
> always safest to rebuild after a hack.
> 
>> If you like, I will be glad to send you the extracts of the error logs. I
>> was having a fine time executing upgrades and patches in Bluelinq and not
>> realizing that they were apparently causing problems. Some patches did not
>> require reboots, and some did, but it was all handled through Bluelinq. I
>> went through several reboots in Bluelinq without hassles, and I was
>> operating with the admin GUI, until the last pass, which was the upgrade
>> from OS 6.1 to 6.2. However, the error logs perhaps show a different tale.
> 
> So now it sounds like you're not saying you were hacked.  No need to
> send me the logs, but perhaps you'll feel comfortable posting a snippet
> here to show us what you mean, so we can all learn.
> 
> Thanks!
> 
> Jeff


Glad to oblige.

Here are messages that I sent to a fellow Listee, the day after the crash:

=================================================================

Hi, Greg-

Thanks a million just for trying.

--Dave

When I perform a shutdown to restart in Telnet, you will see this in the
home-error log (I just did one so as to record what would happen and confirm
that the timestamp is correct:

==========================================================
[Wed Jul 23 13:57:20 2003] [notice] caught SIGTERM, shutting down
[Wed Jul 23 13:59:26 2003] [notice] Apache/1.3.12 Cobalt (Unix) PHP/4.0.4
FrontPage/4.0.4.3 mod_ssl/2.6.6 OpenSSL/0.9.5a mod_perl/1.24 configured --
resuming normal operations
[Wed Jul 23 13:59:26 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin/suexec)

==========================================================

Here are the lines from home-error for the period last night from about 8
p.m. to 3 a.m. when I was doing the updates and at times having to reboot.
(For most of these, I was in the hands of BlueLinq and it chose to reboot
when it felt it needed to.):


===========================================================

[Tue Jul 22 20:16:37 2003] [warn] child process 11591 still did not exit,
sendin
g a SIGTERM
[Tue Jul 22 20:16:37 2003] [warn] child process 23189 still did not exit,
sendin
g a SIGTERM
[Tue Jul 22 20:16:37 2003] [warn] child process 23190 still did not exit,
sendin
g a SIGTERM
[Tue Jul 22 20:16:37 2003] [warn] child process 27279 still did not exit,
sendin
g a SIGTERM
[Tue Jul 22 20:16:37 2003] [warn] child process 27278 still did not exit,
sendin
g a SIGTERM
[Tue Jul 22 20:16:41 2003] [notice] caught SIGTERM, shutting down
:[Tue Jul 22 20:18:41 2003] [notice] Apache/1.3.12 Cobalt (Unix)
PHP/4.0.1pl2 mod
_auth_pam/1.0a FrontPage/4.0.4.3 mod_ssl/2.6.6 OpenSSL/0.9.5a mod_perl/1.24
conf
igured -- resuming normal operations
[Tue Jul 22 20:18:41 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Tue Jul 22 20:20:36 2003] [error] [client 24.136.199.228] File does not
exist: 
/home/groups/home/web/default.ida
[Tue Jul 22 20:26:41 2003] [notice] caught SIGTERM, shutting down
[Tue Jul 22 20:28:45 2003] [notice] Apache/1.3.12 Cobalt (Unix) PHP/4.0.1pl2
mod
_auth_pam/1.0a FrontPage/4.0.4.3 mod_ssl/2.6.6 OpenSSL/0.9.5a mod_perl/1.24
conf
igured -- resuming normal operations
[Tue Jul 22 20:28:45 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Tue Jul 22 20:43:21 2003] [error] [client 24.151.144.28] File does not
exist: /
home/groups/home/web/default.ida
[Tue Jul 22 21:00:47 2003] [error] [client 24.99.4.154] File does not exist:
/ho
me/groups/home/web/default.ida
[Tue Jul 22 21:44:36 2003] [warn] child process 848 still did not exit,
sending 
a SIGTERM
[Tue Jul 22 21:44:36 2003] [warn] child process 849 still did not exit,
sending 
a SIGTERM
[Tue Jul 22 21:44:36 2003] [warn] child process 1599 still did not exit,
sending
 a SIGTERM
:[Tue Jul 22 21:44:41 2003] [notice] caught SIGTERM, shutting down
[Tue Jul 22 21:46:51 2003] [notice] Apache/1.3.12 Cobalt (Unix) PHP/4.0.4
mod_au
th_pam/1.0a FrontPage/4.0.4.3 mod_ssl/2.6.6 OpenSSL/0.9.5a mod_perl/1.24
configu
red -- resuming normal operations
[Tue Jul 22 21:46:52 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Tue Jul 22 22:03:49 2003] [notice] caught SIGTERM, shutting down
[Tue Jul 22 22:06:00 2003] [notice] Apache/1.3.20 Sun Cobalt (Unix)
PHP/4.0.4 mo
d_auth_pam_external/0.1 FrontPage/4.0.4.3 mod_ssl/2.8.4 OpenSSL/0.9.6b
mod_perl/
1.25 configured -- resuming normal operations
[Tue Jul 22 22:06:00 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Tue Jul 22 22:22:37 2003] [warn] child process 848 still did not exit,
sending 
a SIGTERM
[Tue Jul 22 22:22:37 2003] [warn] child process 849 still did not exit,
sending 
a SIGTERM
[Tue Jul 22 22:22:37 2003] [warn] child process 1755 still did not exit,
sending
 a SIGTERM
[Tue Jul 22 22:22:41 2003] [notice] caught SIGTERM, shutting down
[Tue Jul 22 22:25:30 2003] [notice] Apache/1.3.20 Sun Cobalt (Unix)
PHP/4.0.4 mo
d_auth_pam_external/0.1 FrontPage/4.0.4.3 mod_ssl/2.8.4 OpenSSL/0.9.6b
mod_perl/
1.25 configured -- resuming normal operations
[Tue Jul 22 22:25:30 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
:/suexec)
[Tue Jul 22 22:39:27 2003] [warn] child process 851 still did not exit,
sending 
a SIGTERM
[Tue Jul 22 22:39:28 2003] [warn] child process 852 still did not exit,
sending 
a SIGTERM
[Tue Jul 22 22:39:32 2003] [notice] caught SIGTERM, shutting down
[Wed Jul 23 02:52:32 2003] [notice] Apache/1.3.12 Cobalt (Unix) PHP/4.0.4
FrontP
age/4.0.4.3 mod_ssl/2.6.6 OpenSSL/0.9.5a mod_perl/1.24 configured --
resuming no
rmal operations
[Wed Jul 23 02:52:32 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Wed Jul 23 02:59:40 2003] [crit] (98)Address already in use: make_sock:
could n
ot bind to port 80
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = "en",
        LC_ALL = (unset),
        LANG = "en"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
[Wed Jul 23 03:00:08 2003] [crit] (98)Address already in use: make_sock:
could n
ot bind to port 80
perl: warning: Setting locale failed.
:perl: warning: Please check that your locale settings:
        LANGUAGE = "en",
        LC_ALL = (unset),
        LANG = "en"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
[Wed Jul 23 03:00:15 2003] [crit] (98)Address already in use: make_sock:
could n
ot bind to port 80
[Wed Jul 23 03:00:24 2003] [crit] (98)Address already in use: make_sock:
could n
ot bind to port 80
[Wed Jul 23 03:00:58 2003] [notice] caught SIGTERM, shutting down
Apache.pm version 1.27 required!
/usr/lib/perl5/site_perl/5.005/i386-linux/Apache.pm is version 1.26
Perhaps you forgot to 'make install' or need to uninstall an old version?
Found: /usr/lib/perl5/site_perl/5.005/i386-linux/Apache.pm
[Wed Jul 23 03:03:05 2003] [warn] pid file /var/run/httpd.pid overwritten --
Unc
lean shutdown of previous Apache run?
[Wed Jul 23 03:03:07 2003] [notice] Apache/1.3.12 Cobalt (Unix) PHP/4.0.4
FrontP
age/4.0.4.3 mod_ssl/2.6.6 OpenSSL/0.9.5a mod_perl/1.24 configured --
resuming no
rmal operations
[Wed Jul 23 03:03:07 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)

==================================

Hi, Greg--

Here is the activity in adm_error for last night, with the file ending on
the last line you see (in other words, the admin http server never got back
in operation after about 10:22 p.m., even though I had several further
events going on, until 3  a.m.).

--Dave


[Tue Jul 22 19:58:12 2003] [notice] SIGUSR1 received.  Doing graceful
restart
[Tue Jul 22 19:58:23 2003] [notice] Apache/1.3.12 Cobalt (Unix)
mod_ssl/2.6.6 Op
enSSL/0.9.5a PHP/4.0.1pl2 mod_auth_pam/1.0a mod_perl/1.24 configured --
resuming
 normal operations
[Tue Jul 22 19:58:23 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Tue Jul 22 20:16:47 2003] [notice] caught SIGTERM, shutting down
[Tue Jul 22 20:18:39 2003] [notice] Apache/1.3.12 Cobalt (Unix)
mod_ssl/2.6.6 Op
enSSL/0.9.5a PHP/4.0.1pl2 mod_auth_pam/1.0a mod_perl/1.24 configured --
resuming
 normal operations
[Tue Jul 22 20:18:39 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Tue Jul 22 20:23:56 2003] [notice] SIGUSR1 received.  Doing graceful
restart
[Tue Jul 22 20:24:04 2003] [notice] Apache/1.3.12 Cobalt (Unix)
mod_ssl/2.6.6 Op
:enSSL/0.9.5a PHP/4.0.1pl2 mod_auth_pam/1.0a mod_perl/1.24 configured --
resuming
 normal operations
[Tue Jul 22 20:24:04 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Tue Jul 22 20:26:49 2003] [notice] caught SIGTERM, shutting down
[Tue Jul 22 20:28:41 2003] [notice] Apache/1.3.12 Cobalt (Unix)
mod_ssl/2.6.6 Op
enSSL/0.9.5a PHP/4.0.1pl2 mod_auth_pam/1.0a mod_perl/1.24 configured --
resuming
 normal operations
[Tue Jul 22 20:28:41 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Tue Jul 22 20:32:23 2003] [error] ServerScriptHelper.ServerScriptHelper():
Cann
ot authenticate to CCE (login name: , session ID: )
[Tue Jul 22 20:32:23 2003] [error] ServerScriptHelper.ServerScriptHelper():
Cann
ot authenticate to CCE (login name: , session ID: )
ERROR: CCE abruptly disconnected.
ERROR: CCE abruptly disconnected.
ERROR: CCE abruptly disconnected.
ERROR: CCE abruptly disconnected.
ERROR: CCE abruptly disconnected.
ERROR: CCE abruptly disconnected.
[Tue Jul 22 21:44:28 2003] [error] ServerScriptHelper.ServerScriptHelper():
CCE 
is down: A crucial system component (CCE) is currently not operating. Please
try
 again in a moment.
:[Tue Jul 22 21:44:46 2003] [notice] caught SIGTERM, shutting down
[Tue Jul 22 21:46:46 2003] [notice] Apache/1.3.12 Cobalt (Unix)
mod_ssl/2.6.6 Op
enSSL/0.9.5a PHP/4.0.4 mod_auth_pam/1.0a mod_perl/1.24 configured --
resuming no
rmal operations
[Tue Jul 22 21:46:46 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Tue Jul 22 21:48:13 2003] [notice] child pid 776 exit signal Segmentation
fault
 (11)
[Tue Jul 22 21:52:37 2003] [notice] SIGUSR1 received.  Doing graceful
restart
[Tue Jul 22 21:52:44 2003] [notice] Apache/1.3.12 Cobalt (Unix)
mod_ssl/2.6.6 Op
enSSL/0.9.5a PHP/4.0.4 mod_auth_pam/1.0a mod_perl/1.24 configured --
resuming no
rmal operations
[Tue Jul 22 21:52:44 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Tue Jul 22 22:04:00 2003] [notice] caught SIGTERM, shutting down
[Tue Jul 22 22:05:58 2003] [notice] Apache/1.3.20 Sun Cobalt (Unix)
mod_ssl/2.8.
4 OpenSSL/0.9.6b PHP/4.0.4 mod_auth_pam_external/0.1 mod_perl/1.25
configured -- resuming normal operations
[Tue Jul 22 22:05:58 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Tue Jul 22 22:07:07 2003] [notice] child pid 1007 exit signal Segmentation
fault (11)
[Tue Jul 22 22:22:48 2003] [notice] caught SIGTERM, shutting down
:[Tue Jul 22 22:25:29 2003] [notice] Apache/1.3.20 Sun Cobalt (Unix)
mod_ssl/2.8.4 OpenSSL/0.9.6b PHP/4.0.4 mod_auth_pam_external/0.1
mod_perl/1.25 configured -- resuming normal operations
[Tue Jul 22 22:25:29 2003] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin
/suexec)
[Tue Jul 22 22:28:29 2003] [notice] child pid 1101 exit signal Segmentation
faul
t (11)
[Tue Jul 22 22:39:39 2003] [notice] caught SIGTERM, shutting down
(END)