major way to detect malware first top and see if any process use lot of cpu
- 0, first fine offending process using top and find out the user
top //<--find the user and pid of the malware, normally taking up a lot of cpu rm /var/spool/cron/<user> --stop cron from respawn just incase can restore from backup kill -9 pid --this should stop the malware
- cat /proc/pid/pwd to see which dir it run in and see any suspicious files in it and delete it
rm files <-- if no permission, then chmod, if still no permission then lsattr lsattr files <-- check if have is immutable if yes remove it chattr -i -a files <-- remove immutable flags. then rm again rm files
- 2,load up backup and compare files different.
mkdir /mnt/backfil/ <-- make a dir to mount at ls /dev/vd* <-- list the back up device mount /dev/vdc1 /mnt/backfil <-- mount the backup device to file cd / <-- goes to root dir rsync -avn /mnt/backfil/ ./ | less <--compare file differents, n is important testing mode.
- 3,compare the file different and find any suspisious file , e.g, file in etc/, some service executables, copy the file from backup.
- example /etc/ssh/ has been changed , rsynd it from mounts
cd /etc/ssh/ rsync -av /mnt/backfil/etc/ssh/ ./
- 4, determine where the attact came from run
last <-- show when user last login , look for funny ip //notice the login time of the suspicious ip try to map the time in /var/log/secure //scan /var/log/message and /var/log/secure for the suspicious ip //look at how it get in like ssh? cat /var/log/secure cat /var/log/message //then try to find the domain name of the ip nslookup 'suspicious ip'
- 5, a lot of time malware geting via ssh, and once a ssh and sshd was infected, it will log login and passwd and it will infect the next host's ssh
//few way to determin if sshd was infected 1, ls -al //look at the time is recent then it is suspicious, if the host was not installed long time ago. 2, lsattr //look to see if the files are immutable, as sometime malware make it immutable to stop user deleting it.
- 6, look for cpu usage abnormal in www.creativecomputing.com.au/cacti/, compare cpu usage, etc with history data look for abnormal.
Additional firewall rules
We have implemented new firewall rules. This is how our /etc/sysconfig/iptables file looks like:
# Generated by iptables-save v1.4.7 on Mon Feb 6 13:59:27 2023 *filter :INPUT ACCEPT [177:12642] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [184:178154] -A INPUT -s xxx.xxx.0.0/16 -j DROP -A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j DROP -A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,SYN,RST,PSH,ACK,URG -j DROP -A INPUT -s 192.168.0.0/16 -p tcp -m tcp --dport 22 -j ACCEPT -A INPUT -s 10.0.0.0/8 -p tcp -m tcp --dport 22 -j ACCEPT -A INPUT -s yyy.yyy.yyy.yyy/32 -p tcp -m tcp --dport 22 -j ACCEPT -A INPUT -s 127.0.0.0/8 -p tcp -m tcp --dport 22 -j ACCEPT -A INPUT -p tcp -m tcp --dport 22 -j DROP -A OUTPUT -d xxx.xxx.0.0/16 -j DROP COMMIT # Completed on Mon Feb 6 13:59:27 2023
xxx.xxx.0.0/16 is to blacklist certain IP blocks.
the next 2 lines block null packets and "christmas attack" packets.
next, we allow 192.168 and 10. IPs to connect via SSH
yyy.yyy.yyy.yyy we allow head office to connect via SSH
we also allow loopback IPs to connect via SSH
all other ssh packets are dropped.
If file is changed, run the following:
service fail2ban stop service iptables restart service fail2ban start
Places where they have hidden malware:
Here are the places where they have tried to hide malware:
/home/<username>/Desktop/.kthread /root/.kthread /var/tmp/ /tmp/.X11-unix/.kthread
And check the contents of /var/spool/cron (all users, not just root)
to find out if there are hidden ".kthread" files, do:
find / -name .kthread
to try to stop them from coming back using the same method, I have replaced the directory with a 0 length file and then made the file immutable. Example for /tmp/.X11-unix/.kthread
cd /tmp/.X11-unix/ rm -rf .kthread touch .kthread chattr +i .kthread
If they try to create .kthread again, it will fail.