You can move the file around, try moving to an external drive/sd card, then format the drive (or remove it)....

Otherwise for a nagging issue like this, instead of fiddling around, I'll second the Linux option, and Puppy does work well. Although it's been forever since I've deleted files from a booted live usb drive, some may not have write access. Haven't needed to use one for some time other than for installing.

Have a blank usb drive, download something like this http://distro.ibiblio.org/puppylinux/puppy-xenial/64/xenialpup64-7.5-uefi.iso , then download this https://rufus.akeo.ie/ , start it while your usb is plugged in, check to make sure the usb is selected, to the right of "create a bootable disk" the little cd icon, select the file you downloaded, start, and ok/select any default, wait. Then reboot the computer, boot from the drive (varies from system to system, sometimes f11 gives a list, or enter your bios to select boot order). If it doesn't work, I/we can find another one that should, I could test something out.