Some issues to discuss...

Place to discuss Fedora and/or Red Hat
Post Reply
ZiaTioN
administrator
administrator
Posts: 460
Joined: Tue Apr 08, 2003 3:28 pm
Contact:

Some issues to discuss...

Post by ZiaTioN » Tue Oct 04, 2005 8:19 am

Having a little problem with my Linux Box. Little background, I recently purchase a gig of DDR memory for my system and when it arrived I installed it just fine and everything went smoothly. While I had the side cracked open I noticed the side mount fan on my case was disconnected. So being the dumbass I am I did not think that maybe I had it that way in the past for a reason, I hooked it back up.

Now when I booted the system it booted just fine with the new memory but I noticed the side fan was not spinning. I thought "oh well" and simply left it plugged in while not spinning (I know I know). Now I stayed up late last night coding for this project I am working on right now and right before I hit the sack I noticed a low level humming noise coming from "something" in my office. However it was 2 AM and I did not feel like investigating.

Now before I went to bed I started the spider portion of my new project to build the database while I was sleeping. I awoke this morning to find the spider app frozen and the system utterly lagging. I tried to shut everything down gracefully but it was not happening so I was forced to do a hard boot. Now when the system came back up it was running like crap and still showed about 50% of the one gig of memory being used and I thought to myself that is not right. So I did a software reboot via command line. The system rebooted and came back with better memory stats than before so I assumed catastrophy avoided.

Next I went to relaunch my spider app and get a socket error saying it can not bind to 113 (ident). Now some more background here, as part of my spider app I coded a stripped down version of an identd server to satisfy my needs for this app (random changing nick names and such). Now I disabled my real identd server on my box (chkconfig -level 345 identd off) as to allow for my app to use this port. Now I did a quick nmap on the system and saw port 113 was open and listening. I thought that is weird so I tried to stop identd (/etc/init.d/identd stop) but it failed saying it was not running. I checked the chkconfig list (chkconfig --list | grep ident) and indeed it was still showing off for all 6 run levels.

I decided to telnet to port 113 to see what the response was and holy oops it is my spider app detached thread identd server. I was astonished that a few reboots had not killed this thread. So naturally I thought I may need a complete shutdown. This I did attempt but when the system got to the last part where it calls for the ac shutdown of the power supply nothing happened. It stayed right there on that screen until I did a manual shutdown via the power button. Now when I rebooted it lock up a few times and a few more manual shutdowns were required to kick the system into submission.

I finally got the system back up and running but was not satisfied with the stability, oh and by the way my custom ident server is still listening on 113 at this point, so I went and downloaded and installed the latest kernel releases for core 2 thinking I may have hosed the kernel I was using. Once this was complete I rebooted into the new kernel and am seeing the same result.

So in the end I have a detached thread process I can not seem to shake and my system will not shutdown gracefully at all. I think I hosed my power supply with the hole fan thing but am at a loss about the thread. Any ideas on either?

User avatar
Void Main
Site Admin
Site Admin
Posts: 5712
Joined: Wed Jan 08, 2003 5:24 am
Location: Tuxville, USA
Contact:

Post by Void Main » Tue Oct 04, 2005 9:50 am

The first thing I would do is take the memory back out that you added and see if things work as they did. A couple of the problems you mention sound like bad memory. It is normal and good for much of your memory to be shown in use. That doesn't mean that your application is using all that memory but much if it is probably cache. Take a look at /proc/meminfo (cat /proc/meminfo). I think you'll find the more you use your system without rebooting the less free memory you will have but most of it will go to cache which is actually a good thing (higher chance of a data hit in memory rather than having to go to slow disk). Might as well unplug that fan but I don't think it should cause any harm. Running without the extra memory for a while should tell the tale. As far as identd, I don't even have it installed but it should be harmless. Even if I did have it installed it wouldn't run because I always shut off xinetd. You should at least be able to do "chkconfig identd off".

ZiaTioN
administrator
administrator
Posts: 460
Joined: Tue Apr 08, 2003 3:28 pm
Contact:

Post by ZiaTioN » Tue Oct 04, 2005 4:55 pm

Well I monitored my vcore and other voltages via my bios and saw major jumps and knew right away it was a waivering power supply on the way down.

I bought a new 430 watt power supply and everything seems back to normal. The system runs fine and shuts down properly. I do understand the system caching memory but right after a reboot 800 MB of 1 GB should not be in use. The only thing now is I still have this thread to deal with. I am not running the conventional identd as I satted in my first post so shutting it down, which I have done, does no good because it is not running. It is a portion of my application I wrote that starts and detaches a thread to listen on 113. The weird thing is even after a reboot that thread is still detached and listening.

User avatar
Void Main
Site Admin
Site Admin
Posts: 5712
Joined: Wed Jan 08, 2003 5:24 am
Location: Tuxville, USA
Contact:

Post by Void Main » Tue Oct 04, 2005 5:21 pm

So you are saying you don't have an identd process running? There is no way a thread can survive a reboot, what is it that makes you think it does just the PID? If so then if you start it on boot it will always get the same pid. Can I see the output of a "ps auxwww"? Paste the full thing in here and let me know which process you are having trouble with, then the command line used to start that process. If you start no services including your custom app you are saying that immediately after logging in on a reboot you are seeing 800MB in use? Can you post the output of a "cat /proc/meminfo" at this time along with the ps output I asked for?

ZiaTioN
administrator
administrator
Posts: 460
Joined: Tue Apr 08, 2003 3:28 pm
Contact:

Post by ZiaTioN » Tue Oct 04, 2005 6:37 pm

So you are saying you don't have an identd process running?
Correct, I disabled the identd services as to not have a conflick in port bindings.
There is no way a thread can survive a reboot, what is it that makes you think it does just the PID?
Good so I am not crazy because I am under the same impression. What makes me think it does survive a reboot is the fact that I will launch my app, let it run it's course, then reboot. Once the system comes back up there is still a process listening on port 113. So I verify it is not the system identd service by trying to stop it and getting an error. I then check chkconfig --list | grep identd to make sure it is off for all run levels and I also verify nothing in rc.local is starting it.

All that checks out saying identd is not running but there is still a procces bound to that port. I then telnet to "localhost 113" and it connects to the process. I hit enter and the process returns the exact output that my applications custom ident server is suppose to and then closes the telnet connection. This is how I know which process it is.

Here is the output of "ps auxwww"
[root@Freedom root]# ps auxwww
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 1732 456 ? S 19:01 0:01 init [5]
root 2 0.0 0.0 0 0 ? SWN 19:01 0:00 [ksoftirqd/0]
root 3 0.0 0.0 0 0 ? SW< 19:01 0:00 [events/0]
root 4 0.0 0.0 0 0 ? SW< 19:01 0:00 [khelper]
root 16 0.0 0.0 0 0 ? SW< 19:01 0:00 [kacpid]
root 88 0.0 0.0 0 0 ? SW< 19:01 0:00 [kblockd/0]
root 96 0.0 0.0 0 0 ? SW 19:01 0:00 [khubd]
root 150 0.0 0.0 0 0 ? SW 19:01 0:00 [pdflush]
root 151 0.0 0.0 0 0 ? SW 19:01 0:00 [pdflush]
root 153 0.0 0.0 0 0 ? SW< 19:01 0:00 [aio/0]
root 152 0.0 0.0 0 0 ? SW 19:01 0:00 [kswapd0]
root 246 0.0 0.0 0 0 ? SW 19:01 0:00 [kseriod]
root 441 0.0 0.0 0 0 ? SW 19:01 0:00 [kjournald]
root 1471 0.0 0.0 0 0 ? SW 19:01 0:00 [kjournald]
root 2195 0.0 0.0 2784 596 ? S 19:01 0:00 syslogd -m 0
root 2199 0.0 0.0 2808 444 ? S 19:01 0:00 klogd -x
rpc 2220 0.0 0.0 2640 572 ? S 19:01 0:00 portmap
rpcuser 2240 0.0 0.0 1824 704 ? S 19:01 0:00 rpc.statd
root 2278 0.0 0.0 2356 560 ? S 19:01 0:00 rpc.idmapd
root 2382 0.0 0.0 2496 740 ? S 19:01 0:00 /usr/sbin/smartd
root 2392 0.0 0.0 3436 460 ? S 19:01 0:00 /usr/sbin/acpid
root 2404 0.0 0.2 8140 2116 ? S 19:01 0:00 cupsd
root 2542 0.0 0.1 5124 1448 ? S 19:01 0:00 /usr/sbin/sshd
root 2557 0.0 0.0 3004 884 ? S 19:01 0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
ntp 2581 0.0 0.5 5320 5320 ? SL 19:01 0:00 ntpd -U ntp -p /var/run/ntpd.pid
root 2595 0.0 0.1 4508 1140 ? S 19:01 0:00 /bin/sh /usr/bin/safe_mysqld --defaults-file=/etc/my.cnf --pid-file=/var/run/mysqld/mysqld.pid
mysql 2624 0.0 0.4 30568 5140 ? S 19:01 0:00 /usr/libexec/mysqld --defaults-file=/etc/my.cnf --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking
root 2650 0.0 0.2 7696 2752 ? S 19:01 0:00 sendmail: accepting connections
smmsp 2659 0.0 0.2 6180 2348 ? S 19:01 0:00 sendmail: Queue runner@01:00:00 for /var/spool/clientmqueue
root 2670 0.0 0.0 1848 464 ? S 19:01 0:00 gpm -m /dev/input/mice -t imps2
root 2681 0.0 1.1 24136 11912 ? S 19:01 0:00 /usr/sbin/httpd
root 2692 0.0 0.0 2104 636 ? S 19:01 0:00 crond
apache 2714 0.0 1.1 24136 11936 ? S 19:01 0:00 /usr/sbin/httpd
apache 2715 0.0 1.1 24136 11932 ? S 19:01 0:00 /usr/sbin/httpd
apache 2716 0.0 1.1 24136 11932 ? S 19:01 0:00 /usr/sbin/httpd
apache 2717 0.0 1.1 24136 11932 ? S 19:01 0:00 /usr/sbin/httpd
apache 2718 0.0 1.1 24136 11932 ? S 19:01 0:00 /usr/sbin/httpd
apache 2719 0.0 1.1 24136 11932 ? S 19:01 0:00 /usr/sbin/httpd
apache 2720 0.0 1.1 24136 11932 ? S 19:01 0:00 /usr/sbin/httpd
apache 2721 0.0 1.1 24136 11932 ? S 19:01 0:00 /usr/sbin/httpd
xfs 2722 0.0 0.2 5060 3104 ? S 19:01 0:00 xfs -droppriv -daemon
root 2732 0.0 0.2 11264 2408 ? S 19:01 0:00 smbd -D
root 2736 0.0 0.1 8060 2004 ? S 19:01 0:00 nmbd -D
daemon 2755 0.0 0.0 3108 596 ? S 19:01 0:00 /usr/sbin/atd
root 2769 0.0 0.2 11264 2396 ? S 19:01 0:00 smbd -D
dbus 2775 0.0 0.0 3444 804 ? S 19:01 0:00 dbus-daemon-1 --system
root 2784 0.0 0.2 8916 2124 ? S 19:01 0:00 /sbin/mount.smbfs //Heckyl/D /mnt/HECKYL_D -o rw username XXX password XXXXXXXX
root 2798 0.0 0.0 0 0 ? SW 19:01 0:00 [smbiod]
root 2801 0.0 0.1 8760 2028 ? S 19:01 0:00 /sbin/mount.smbfs //Heckyl/E /mnt/HECKYL_E -o rw username XXX password XXXXXXXX
root 2805 0.0 0.2 8864 2124 ? S 19:01 0:00 /sbin/mount.smbfs //Jeckyl/D /mnt/JECKYL_D -o rw username XXX password XXXXXXXX
root 2809 0.0 0.2 7856 2128 ? S 19:01 0:00 /sbin/mount.smbfs //Lapdog/D /mnt/LAPDOG_D -o rw username XXX password XXXXXXXX
root 2812 0.0 0.1 4684 1884 ? S 19:02 0:00 ssh -l ethompson -g -C -N -f -L 23:localhost:22 some_site.com
root 2819 0.0 0.0 2792 324 ? S 19:02 0:00 mdadm --monitor --scan
root 2849 0.0 0.0 1704 344 tty1 S 19:02 0:00 /sbin/mingetty tty1
root 2855 0.0 0.0 3244 340 tty2 S 19:02 0:00 /sbin/mingetty tty2
root 2861 0.0 0.0 2620 344 tty3 S 19:02 0:00 /sbin/mingetty tty3
root 2867 0.0 0.0 2696 340 tty4 S 19:02 0:00 /sbin/mingetty tty4
root 2873 0.0 0.0 1708 340 tty5 S 19:02 0:00 /sbin/mingetty tty5
root 2874 0.0 0.0 2704 344 tty6 S 19:02 0:00 /sbin/mingetty tty6
root 2880 0.0 0.2 11152 2096 ? S 19:02 0:00 /usr/bin/gdm-binary -nodaemon
root 3068 0.0 0.2 11684 2568 ? S 19:02 0:00 /usr/bin/gdm-binary -nodaemon
root 3079 0.0 0.7 11464 8200 ? S 19:02 0:00 /usr/X11R6/bin/X :0 -audit 0 -auth /var/gdm/:0.Xauth -nolisten tcp vt7
gdm 3108 0.0 0.8 20864 8820 ? S 19:02 0:00 /usr/bin/gdmgreeter
root 3109 0.0 0.2 8764 2084 ? S 19:02 0:00 sshd: root@pts/1
root 3111 0.0 0.1 5580 1364 pts/1 S 19:02 0:00 -bash
root 3169 0.0 0.0 3676 744 pts/1 R 20:30 0:00 ps auxwww
f you start no services including your custom app you are saying that immediately after logging in on a reboot you are seeing 800MB in use?
This was the case before I replaced the power supply. Now all issues with system performance have been resolved by a new power supply.
Can you post the output of a "cat /proc/meminfo" at this time along with the ps output I asked for?
I can if this would help with the thread issue but if you are asking because of the old memory issue I had before the power supply swap then it may not be needed since I do not have that issue anymore.

ZiaTioN
administrator
administrator
Posts: 460
Joined: Tue Apr 08, 2003 3:28 pm
Contact:

Post by ZiaTioN » Tue Oct 04, 2005 6:40 pm

Here is also some output of the checking I do to verify if something is actually listening:
[root@Freedom root]# nmap 127.0.0.1

Starting nmap 3.93 ( http://www.insecure.org/nmap/ ) at 2005-10-04 20:37 EDT
Interesting ports on Freedom (127.0.0.1):
(The 1656 ports scanned but not shown below are in state: closed)
PORT STATE SERVICE
22/tcp open ssh
23/tcp open telnet
25/tcp open smtp
111/tcp open rpcbind
113/tcp open auth
139/tcp open netbios-ssn
443/tcp open https
445/tcp open microsoft-ds
631/tcp open ipp
3306/tcp open mysql
32770/tcp open sometimes-rpc3
32771/tcp open sometimes-rpc5

Nmap finished: 1 IP address (1 host up) scanned in 0.148 seconds
[root@Freedom root]# /etc/init.d/identd stop
Stopping identd services: [FAILED]
[root@Freedom root]# chkconfig --list | grep identd
identd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[root@Freedom root]# telnet localhost 113
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

: USERID : UNIX : hostname3183Connection closed by foreign host.
[root@Freedom root]#
This is very strange.
Last edited by ZiaTioN on Mon Oct 31, 2005 12:22 pm, edited 1 time in total.

ZiaTioN
administrator
administrator
Posts: 460
Joined: Tue Apr 08, 2003 3:28 pm
Contact:

Post by ZiaTioN » Tue Oct 04, 2005 6:47 pm

I found the process, 2557.
root 2557 0.0 0.0 3004 884 ? S 19:01 0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
However why is this running? I never start it, or am I unaware of another startup method?

User avatar
Void Main
Site Admin
Site Admin
Posts: 5712
Joined: Wed Jan 08, 2003 5:24 am
Location: Tuxville, USA
Contact:

Post by Void Main » Wed Oct 05, 2005 4:54 am

As I mentioned before xinetd is the what spawns identd. What is the output of the "checkconfig --list" command? What is in your /etc/xinetd.d directory? xinetd replaced inetd which is what used to spawn processes like telnetd, fingerd, ftpd, tftpd, etc. None of those processes ran all the time. inetd would listen on those ports for a connection request and when a spawn the appropriate service when needed. xinetd now does this. That's why I asked if it was running. It's probably in your list of services (chkconfig --list). Just turn it off if you don't use any of the processes it spawns (I always turn it off).

# chkconfig xinetd off
# service xinetd stop
# nmap localhost

Post Reply