My server has been hacked.

[Void Main]

Did you do the "ps auxwww > /tmp/ps.lst" and upload it? I see the find file, but not the ps.

EDIT: Disregard, I see it. :)

[Void Main]

The only one that stands out to me is PID 901 (sk) but you are more familiar with your system. Do you see processes running that you can't explain? I gotta hit the sack for tonight though. I can't keep my eyes open... L8r

[Webdiggity]

Can I dump the output of the u (uninstall command) to a file too?

Just wondering.

[Webdiggity]

root@dig2 [/usr/include/sdk386]# ./sk u
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* SUCKIT SUCKIT *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Currently installed version: v1.1c
Removing from memory...OK, previous call dispatch 0xc02dc9d8 at 0xc01088bf restored.


This is the last thing I did. Any thoughts?

Am I safe now?

[Webdiggity]

root@dig2 [/usr/include/sdk386]# ./sk u
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* SUCKIT SUCKIT *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Currently installed version: v1.1c
Removing from memory...OK, previous call dispatch 0xc02dc9d8 at 0xc01088bf restored.

This is the last thing I ran.

Do you think I'm safe now?

[Void Main]

No, I don't think you are safe. Did you look over your process list and account for everything that is running?

Also that "ev" command is an ELF binary parasite. I don't know if he ran the command on any of your files and I am not exactly sure what it actually does but it does attach some code to an ELF executable (this is possibly the first virus I have ever witnessed first hand in over 10 years in Linux, although it appears to have been manually placed). The code that it attaches is in the source file as binary so the instructions would have to be translated into assembler and interpereted. I can't find this source in Google but if the date stamp on the file is correct it has been around for almost a year (Mar 6 2003). I don't really want to make it available for public download but I believe it needs to be sent in to CERT and SANS, etc. I didn't come up with anything on a search there either. Now, I just wrote a script that should be able to scan your system and detect any files that contain this parasite. I believe it should work. You can grab the script here:

http://voidmain.is-a-geek.net/files/scr ... ndparasite

Download it, set it executable (chmod +x findparasite) and then scan your entire system by:

Code: Select all

# ./findparasite / > /tmp/infected.lst


I only included a portion of the binary data to search for from the actual source. Let me know if anything ends up in your infected.lst file. Actually before scanning the entire system you might want to do some test runs on all your bin directories (/bin, /usr/bin, /sbin, /usr/sbin, /usr/local/bin, etc). I don't know if certain special files (device files, etc) will screw up the scan, or make it take an unbearably long time. I suppose I could check that myself now.

However, it really doesn't matter what was put on your system, none of that tells us *how* the person got in to your system and gained root privileges in the first place. I would say you definitely need to rebuild your system. All of your web content should be safe, except for possibly any binary executables (any PHP, Perl, HTML, etc should be ok). It would be nice if the drive could be removed and kept in the state it is in now and only accessed in a read-only way to try and figure out how the person got it.

Without finding out "how" the person got in you certainly can't be sure that they can't do it again, even after rebuilding your system. If you expose *any* piece of what is used on your old system then it may contain the vulnerability. It's also possible that they gained a password from sniffing a telnet/ftp session. It would be wise to have *everyone* have different passwords on your new system and to use encrypted protocols if at all possible (no ftp or telnet). And any management you may do though a web interface should be done via SSL (https).

I am going to see what I can do about reporting this to CERT. They may or may not be interested in it. In fact, might I suggest that *you* actually be the one to report it? Here is the CERT reporting page:

https://irf.cc.cert.org/

Feel free to give them the link to this thread so they can see what we have already done and found. You might also mention that the installed kit includes a parasite (ELF virus), including source, that we have not been able to find in any google searches.

EDIT: I have disassembled the included binary data from the source code and am now trying to interpret it. I'm a little rusty on my assembly so we'll see how it goes. :)

[Webdiggity]

I'm nearly done copying the accounts only the /home accounts.

When I'm done I'll let you poke around in there if you want. I really won't have a need for it.

I killed that process last night and uninstalled it per the sk u command.

This morning it was back.

Yikes. We may have stumbled onto something eh? Is there a reward? hahahaha

[Webdiggity]

Running the find parasite script now.

[Void Main]

I hope it's still not running. If it is kill it and run on individual directories "/usr", "/home", "/var", "/etc", "/tmp", "/dev", etc. Do not run it on "/proc", maybe that's where it's hanging. You could output to a separate log for each directory you run it on. And yes, after you get your users migrated off I wouldn't mind getting in and having you give me root access to look around. I would very much like to find out how they got in.

[Webdiggity]

I pm'ed you the login that I rarely use for SSH.

[Void Main]

Just to recap a little more for anyone who might have been following along. There was a program included with this kit that I thought looked like a virus but after looking at it some more I don't believe it is a virus. It is called a parasite and you can run this command to inject a bit of binary code into an ELF executable.

The reason I say it's not a virus is because it does not appear to in turn infect other ELF executables when the infected executable is run. I had set up a virtual test linux system (using "user mode linux") and put the kit on it, exected the parasite executable on something in /usr/bin to inject the parasite. I then run the infected file and nothing seems to happen.

Of course I am missing a *very* key piece of information in this whole test. Just what exactly is the small amount of binary that gets injected into the ELF executables? I disassebled the binary but I still haven't interpereted what the small amount of ASM is supposed to do. If anyone knows of someone who might be able to make sense of it let me know so I can get it to them. We're talking about less than 100 bytes of binary code. I do know that it uses some functions of Linux "INT 0x80". The funny thing was, it didn't appear that anything on the cracked system was actually infected with said parasite. Maybe it was just part of the kit the cracker used but didn't actually get used.

[Webdiggity]

I would like to add that I'll have this server until the end of the month so Void, if you have someone that you trust who wants a peek at the box, let me know. After the 28th this server turns back into a pumpkin.

[Void Main]

Nope, I don't know of anyone. It just bugs me not knowing, but not enough to overcome my laziness. :) You might want to let the Server Beach people know about it, they might be interested since it might have been their product that was exploited.

[Webdiggity]

I did some digging in my control panel and the updates were set to automatically upgrade to the next stable releases on the system software and control panel software so I don't think they would have gotten in that way.

Did you get the E-mail I sent you that I received from a customer. I sent it in PM. Didn't know if that might help.

There is a pretty sharp guy who works for ServerBeach and also runs their boards so I might tell him. I'm done with that box and I'm digging into the rsync and security docs now so I can make sure I have rock solid incremental backups.

Thanks for everything. I still owe ya lunch.

[Void Main]

Well, I meant that Server Beach might be interested if this is a newly discovered vulnerability in their software (that they don't know about) which is the only reason they would be interested. If you had a really old version of the software that had known vulnerabilities then they probably wouldn't be too interested. :)

No, I didn't get a PM regarding any email. Unless you sent it to me in an email, if so it musta been to an account I haven't read yet today. I'll check them all now.