How to guarantee files that are decrypted during run time are cleaned up?

Using C or C++, After I decrypt a file to disk- how can I guarantee it is deleted if the application crashes or the system powers off and can't clean it up properly? Using C or C++, on Windows and Linux?

From stackoverflow

• Try to avoid it completely:

  If the file is sensitive, the best bet is to not have it written to disk in a decrypted format in the first place.

  Protecting against crashes: Structured exception handling:

  However, you could add structured exception handling to catch any crashes.

  __try and __except

  What if they pull the plug?:

  There is a way to protect against this...

  If you are on windows, you can use MoveFileEx and the option MOVEFILE_DELAY_UNTIL_REBOOT with a destination of NULL to delete the file on the next startup. This will protect against accidental computer shutdown with an undeleted file. You can also ensure that you have an exclusively opened handle to this file (specify no sharing rights such as FILE_SHARE_READ and use CreateFile to open it). That way no one will be able to read from it.

  Other ways to avoid the problem: All of these are not excuses for having a decrypted file on disk, but:

  • You could also consider writing to a file that is larger than MAX_PATH via file syntax of \\?\. This will ensure that the file is not browsable by windows explorer.

  • You should set the file to have the temporary attribute

  • You should set the file to have the hidden attribute

• A process cannot protect or watch itself. Your only possibility is to start up a second process as a kind of watchdog, which regularly checks the health of the decrypting other process. If the other process crashes, the watchdog will notice and delete the file itself.

  You can do that using hearth-beats (regular polling of the other process to see whether it's still alive), or using interrupts sent from the other process itself, which will trigger a timeout if it has crashed.

  You could use sockets to make the connection between the watchdog and your app work, for example.

  It's becoming clear that you need some locking mechanism to prevent swapping to the pagefile / swap-partition. On Posix Systems, this can be done by the m(un)lock* family of functions.

  Martin York : hearth-beats! Is this like checking the fire is still going? :-)

• I don't know if this works on Windows, but on Linux, assuming that you only need one process to access the decrypted file, you can open the file, and then call unlink() to delete the file. The file will continue to exist as long as the process keeps it open, but when it is closed, or the process dies, the file will no longer be accessible.

  Of course the contents of the file are still on the disk, so really you need more than just deleting it, but zeroing out the contents. Is there any reason that the decrypted file needs to be on disk (size?). Better would just to keep the decrypted version in memory, preferably marked as unswappable, so it never hits the disk.

  Graeme Perrow : FYI, no you cannot do this on Windows.

  mpez0 : If you decrypt to a RAM based file system (that is, not actually on disk) then a crash or process failure will have all all the data actually gone.

  KeithB : mpez0, thats a good point. As long as you can ensure that it won't be swapped out, everything is fine.

• This is a tricky topic. Generally, you don't want to write decrypted files to disk if you can avoid it. But keeping them in memory doesn't always guarentee that they won't be written to disk as part of a pagefile or otherwise.

  I read articles about this a long time ago, and I remember there being some difference between Windows and Linux in that one could guarentee a memory page wouldn't be written to disk and one couldn't; but I don't remember clearly.

  If you want to do your due diligence, you can look that topic up and read about it. It all depends on your threat model and what you're willing to protect against. After all, you can use compressed air to chill RAM and pull the encryption key out of that (which was actually on the new Christian Slater spy show, My Own Worst Enemy - which I thought was the best use of cutting edge, accurate, computer security techniques in media yet)

• Unfortunately, there's no 100% foolproof way to insure that the file will be deleted in case of a full system crash. Think about what happens if the user just pulls the plug while the file is on disk. No amount of exception handling will protect you from that (the worst) case.

  The best thing you can do is not write the decrypted file to disk in the first place. If the file exists in both its encrypted and decrypted forms, that's a point of weakness in your security.

  The next best thing you can do is use Brian's suggestion of structured exception handling to make sure the temporary file gets cleaned up. This won't protect you from all possibilities, but it will go a long way.

  Finally, I suggest that you check for temporary decrypted files on start-up of your application. This will allow you to clean up after your application in case of a complete system crash. It's not ideal to have those files around for any amount of time, but at least this will let you get rid of them as quickly as possible.

  Brian R. Bondy : Also locking the file is a good idea, assuming not writing the file in the first place is not an option for him.

  Bill the Lizard : That's a good point. Locked and hidden.

  n-alexander : unlinking the file works with sudden system crashes

• Don't write the file decrypted to disk at all.

  If the system is powerd off the file is still on disk, the disk and therefore the file can be accessed.

  Exception would be the use of an encrypted file system, but this is out of control of your program.

  erickson : Good point about encrypted file systems.

  Klathzazt : This was actually one of the things I was considering: creating and using an encrypted volume as temp file space while the application is running.

• Check out tmpfile().

  It is part of BSD UNIX not sure if it is standard.
  But it creates a temporary file and automatically unlinks it so that it will be deleted on close.

  Writing to the file system (even temporarily) is insecure.
  Do that only if you really have to.

  Optionally you could create an in-memory file system.
  Never used one myself so no recommendations but a quick google found a few.

• There's a problem with deleting the file. It's not really gone. When you delete files off your hard drive (not counting the recycle bin) the file isn't really gone. Just the pointer to the file is removed.

  Ever see those spy movies where they overwrite the hard drive 6, 8,24 times and that's how they know that it's clean.. Well they do that for a reason. I'd make every effort to not store the file's decrypted data. Or if you must, make it small amounts of data. Even, disjointed data.

  If you must, then they try catch should protect you a bit.. Nothing can protect from the power outage though.

  Best of luck.

• In C (and so, I assume, in C++ too), as long as your program doesn't crash, you could register an atexit() handler to do the cleanup. Just avoid using _exit() or _Exit() since those bypass the atexit() handlers.

  As others pointed out, though, it is better to avoid having the decrypted data written to disk. And simply using unlink() (or equivalent) is not sufficient; you need to rewrite some other data over the original data. And journalled file systems make that very difficult.

• In C++ you should use an RAII tactic:

  class Clean_Up_File {
      std::string filename_;
      public Clean_Up_File(std::string filename) { ... } //open/create file
      public ~Clean_Up_File() { ... } //delete file
  }
  
  int main()
  {
      Clean_Up_File file_will_be_deleted_on_program_exit("my_file.txt");
  }
  

  RAII helps automate a lot of cleanup. You simply create an object on the stack, and have that object do clean up at the end of its lifetime (in the destructor which will be called when the object falls out of scope). ScopeGuard even makes it a little easier.

  But, as others have mentioned, this only works in "normal" circumstances. If the user unplugs the computer you can't guarantee that the file will be deleted. And it may be possible to undelete the file (even on UNIX it's possible to "grep the harddrive").

  ---

  Additionally, as pointed out in the comments, there are some cases where objects don't fall out of scope (for instance, the std::exit(int) function exits the program without leaving the current scope), so RAII doesn't work in those cases. Personally, I never call std::exit(int), and instead I either throw exceptions (which will unwind the stack and call destructors; which I consider an "abnormal exit") or return an error code from main() (which will call destructors and which I also consider an "abnormal exit"). IIRC, sending a SIGKILL also does not call destructors, and SIGKILL can't be caught, so there you're also out of luck.

  Martin York : Does not help on abnormal exit. exit() abort() terminate() never return and do not unwind the call stack so RAII does not happen.

  Max Lybbert : I'll edit the question; however it looks like we're using different definitions of "abnormal exit."

• on Linux/Unix, use unlink as soon as you created the file. The file will be removed as soon as you program closes the file descriptor or exits.

  Better yet, the file will be removed even if the whole system crashes - because it is basically removed as soon as you unlink it.

  The data will not be physically deleted from the disk, of course, so it still may be available for hacking.

• Remember that the computer could be powered down at any time. Then, somebody you don't like could boot up with a Linux live CD, and examine your disk in any level of detail desired without changing a thing. No system that writes plaintext to the disk can be secure against such attacks, and they aren't hard to do.

  You could set up a function that will overwrite the file with ones and zeros repeatedly, preferably injecting some randomness, and set it up to run at end of program, or at exit. This will work, provided there are no hardware or software glitches, power failures, or other interruptions, and provided the file system writes to only the sectors it claims to be using (journalling file systems, for example, may leave parts of the file elsewhere).

  Therefore, if you want security, you need to make sure no plaintext is written out, and that also means it cannot be written to swap space or the equivalent. Find out how to mark memory as unswappable on all platforms you're writing for. Make sure decryption keys and the like are treated the same way as plaintext: never written to the disk under any circumstances, and kept in unswappable memory.

  Then, your system should be secure against attacks short of hostiles breaking in, interrupting you, and freezing your RAM chips before powering down, so they don't lose their contents before being transferred for examination. Or authorities demanding your key, legally (check your local laws here) or illegally.

  Moral of the story: real security is hard.

• The method that I am going to implement will be to stream the decryption- so that the only part that is in memory is the part that is decrypted during the read as the data is being used. Here is a diagram of the pipeline:

  This will be a streamed implementation, so the only data that is in memory is the data that I am consuming in the application at any given point. This makes some things tricky- considering a lot of traditional file tricks are no longer available, but since the implementation will be stream based i will still be able to seek to different points of the file which would be translated to the crypt stream to decrypt at different sections.

  Basically, it will be encrypting blocks of the file at a time - so then if I try to seek to a certain point it will decrypt that block to read. When I read past a block it decrypts the next block and releases the previous (within the crypt stream).

  This implementation does not require me to decrypt to a file or to memory and is compatible with other stream consumers and providers (fstream).

  This is my 'plan'. I have not done this type of work with fstream before and I will likely be posting a question as soon as I am ready to work on this.

  Thanks for all the other answers- it was very informative.