I’ve seen it before, a customer deletes a file and then needs it restored.
Normally a challenging request, but under special circumstances a process may have the file opened.
While showing my son some fun and exciting Linux security scenarios, I recalled all those times I was able to recover data from the /proc (in memory) filesystem. In order to visualize this scenario I threw together a small Dockerfile and had him poke around:
Once logged in you notice a server.py script, for our scenario this is our companies server daemon. If we browse the code we notice a file named password.txt was opened,
but never closed
. Ignore the sleep() function, this is just here to keep the file opened, and to simulate a daemon process. The next things you should do it verify the server is actively running! If we hope to recover that file the server needs to have a lock on password.txt
root@4df66824ad14:~# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 18504 3396 pts/0 Ss 01:37 0:00 /bin/bash
root 10 0.0 0.1 22548 6076 pts/0 S 01:37 0:00 python server.py
root 24 0.0 0.0 34396 2904 pts/0 R+ 01:45 0:00 ps aux
We are in luck!
From here we can use the lsof command to verify password.txt is still opened:
root@4df66824ad14:~# lsof | grep password.txt
python 10 root 3r REG 8,1 30 15860072 /root/password.txt (deleted)
Bingo! we see the file is opened, and it is marked for deletion. Good thing our service is still running!
All that is left is to track down our service’s process id (we see above it’s process id, or PID is 10).
We now have all the pieces to track down our password file from the in memory filesystem.
The /proc filesystem has a number of useful files, it also includes a directory for every running process (named after the PID).
Within the process directory we have a slew of files; however, the directory we care about is fd (standing for file descriptor):
root@4df66824ad14:~# cd /proc/10
root@4df66824ad14:/proc/10# ls -l fd
total 0
lr-x------ 1 root root 64 Sep 28 01:43 0 -> /dev/null
lrwx------ 1 root root 64 Sep 28 01:43 1 -> /dev/pts/0
lrwx------ 1 root root 64 Sep 28 01:43 2 -> /dev/pts/0
lr-x------ 1 root root 64 Sep 28 01:43 3 -> '/root/password.txt (deleted)'
And just like that we found a file representing our in memory password.txt file!
From here we can show its contents, or even restore it using cp:
root@4df66824ad14:/proc/10# cat fd/3
the-most-secret-password-ever
Happy hacking!