From a Microsoft support document (as discovered by Neowin):
Windows defines two main policies, Quick removal and Better performance, that control how the system interacts with external storage devices such as USB thumb drives or Thunderbolt-enabled external drives. Beginning in Windows 10 version 1809, the default policy is Quick removal.
In earlier versions of Windows the default policy was Better performance.
What this means is that starting with Windows 10 version 1809, you no longer need to use the Safely Remove Hardware process when removing a USB drive, because there’s no longer any write operation caching going on. You can still change this policy if you want to.
This is a dubious claim. USB mass storage devices should support many different commands, including things like “start/stop unit”, allow media removal, … I don’t know which commands are issued before removing an USB device, but warning that the power is about to be cut cannot be a bad thing. Maybe it doesn’t change much on simple USB storage, but large capacity units, enventualy in USB to nVME or SATA enclosures, can use sophisticated background algorithms such as compaction of erazed areas, caching blocks to different flash areas… Things that it would be better not interrupt without prior warning.
Yanking the power on any drive is a bad idea. Want to bet that the people who’re say doing this is a good idea are going to be nowhere to be found when the data on the usb drive gets scrambled?
There’s “IT people” (e.g. us) that know why “safely remove hardware” exists and do the right thing; and these people will mostly think it’s a bad idea (for reliability and performance).
However; there’s also “not IT people”. I’ve seen too many people just rip devices out without caring. Most have been using Windows for years and still don’t even realize you’re supposed to do the “safely remove hardware” thing. For these people, I think Microsoft’s change is appropriate (in a “choose the lesser of 2 evils” way).
And then there are those of us that understand why that option exists and still remove the USB stick without using it, with 0% failure since Windows NT started supporting USB. Just need to understand when it is “safe” to do so and knowing that even Microsoft doesn’t screw up that often. Note that I don’t do this with USB connected HDD adapters and such, just raw storage sticks.
Lucky. I literally told someone it wasn’t really really necessary to do the safely remove drive thing, Then demonstrated with their drive on their computer … and totally corrupted the drive. It wasn’t blinking or anything, files had only been copied from the device onto the computer ( all of which were correctly saved to the computer). Just the drive was borked, couldn’t read as it was, couldn’t reformat, it was just screwed. And we were 50 miles from the closest place to buy a replacement that was really needed. So, I had to do a 100 mile round trip, and shell out for the $100 replacement ( this was back in 2003-2004 or so). It sucked. All because I wanted to save a few seconds and some tedious mouse clicking.
Bill Shooter of Bul
This is a pretty big problem with nand flash, sudden power loss can even kill an SSD. The paper doesn’t identify manufactures, but both Intel and Crucial SSDs have been killable via ill-timed power loss in the past.
https://www.usenix.org/system/files/conference/fast13/fast13-final80.pdf
Long story short, NAND flash requires far more extensive use of metadata to implement things like dynamic wear-leveling and program/erase cycles, the erase portion happening in the background when the device is otherwise idle. Metadata needs to get updated for every operation to keep track of where blocks are located and where they can be written to, their remaining endurance life, etc. This is constantly getting updated. The potential for data loss from corrupt metadata is high, and the frequency of metadata updates is also high.
With spinning HDDs, there’s far less meta-data complexity to begin with. There are fewer background processes, metadata updates are relatively rare, and writing data during a loss of power results in an incomplete write, which is unlikely to be catastrophic for a HDD.
Hopefully most SSD/flash manufactures have stepped up their game in terms of addressing metadata corruption during power loss, intel apparently has done so. But having experienced catastrophic SSD failure myself two years ago (and usb/sd flash card failures before that), I’m still nervous about it. I’ve decided the risk warrants running SSDs in RAID, however since both SSDs will experience a power failure, I’m not sure how much good this does. Two identically bricked SSDs defeats RAID. As a precaution, I do recommend using one SSD for a little while before joining it to the RAID to help ensure the SSD metadata/state machines are out of sync to decrease the odds of simultaneous failure (hopefully).
I’ve been running on UPS power backup for decades now, which provides some peace of mind, but sometimes during a crash I have no choice but to do an instantaneous “hard” reboot / shutdown. I have no clue if the duration of power loss matters. On servers with battery-backup controllers there’s less to worry about since they should be able to shutdown cleanly regardless of external factors. Now if only they could do something about the limited SSD write endurance…oh the joys of technology, haha.
Are you suggesting my flash drives secretly doing destructive data operations, invisible to the operating system, that could damage my data or the device itself, such that unexpected power loss can cause me to lose data or damage my device?
What kind of bonehead would design NVM like this?
Yes, they do. Things like garbage collection, refreshing cells and more and more complex things as flash densities gets higher and more bits are squeezed into fewer transistors.
There are many providers of special “embedded” mass storage, SATA drives, SD cards, and USB devices optimized to better sustain transient or untimely power cuts (and extended temperature ranges, vibrations, military encryption, etc). They use non-volatile memory such as MRAM for journaling, and/or use capactitors banks to give extra time for proper shutdown. Of course they are usually more expensive, and slower than mass storage for servers which can aggressively cache data, delay writes.
This is going to make things like external thunderbolt raid arrays and even SAS controllers that present as hot pluggable a lot slower (by default) by the sounds of it. And it’ll still likely lead to damaged filesystems anyway. Great.
The issue here is not power, but the OS disk cache: the UI shows the write to a USB are finished when actually the application finished writing to the file system cache. This cache is still to be flushed to the USB stick.
That’s how many damaged their files on the USB sticks, by removing them before the actual writing is done. What this Windows change does, is disabling the write cache so that the UI shows the actual speed and completion.
This settings can be changed on any Windows, at least since Win95.
Top part of this window: https://blog.vmpros.nl/wp-content/uploads/2010/03/image10.png
Yeah, but it’s a mess to find, when a setting panel’s entry could have been added long before.
It’s pretty easy to find. Just right-click on the device in Device Manager.
Not sure how it could get any easier, considering how niche of a setting it is. It is exactly where my first guess was.
ChodaSly,
You’re right that the OS has to finish writing it’s cache, however it’s wrong to discredit the power issue. Flash drives perform internal operations (wear leveling and erase cycles) that are not normally visible to the OS. It’s not necessarily safe to unplug a flash drive while it’s performing these operations even though the OS may have completed writing to disk! It’s true that ejecting flash drives tells windows to flush the cache, but it additionally tells the flash drive to turn off.
If you’ve got a flash drive with a power light on it, you can see this for yourself. The light comes on when you plug it in. You can read & write to the media, which typically causes the light to flash, but it remains on meaning that it’s internal NAND controller continues to run in the background. Even if you write absolutely nothing to the disk, the flash controller may perform background functions. Now when you virtually eject the drive in windows, not only is any remaining cache written the drive but additionally it powers down and the light turns off. It’s not until this state that the flash drive is guaranteed to be safe to remove. If you fail to do this, you risk corrupting the flash drive’s hidden metadata that keeps track internal flash block mapping & accounting.
Ideally all media would be impervious to this, but alot media is not and there is a risk of metadata corruption when power is removed without notice.
You may or may not have noticed that hard disks have a shutdown function as well. On a normal shutdown, the OS tells all disks to power down, and they spin down & park gracefully. However if you do a “hard” reboot/poweroff, HDDs will clank the head assembly to hastily park it since an uncontrolled & unparked head can scratch the platters.
Hm, AFAIK it will park the head as “hastily” as always, but using the kinetic energy of the spinning platters and the motor “in reverse”, as a generator to supply power for the operation?… (quite brilliantly simple 🙂 )
The comments here makes me think the physical usability design may just be the wrong design. Like the CD drives of old, we need physical safeguards against incorrect removal. A physical eject button is needed, and tells the OS to non-negotiably flush all caches, finish writes or cancel in a timely and safe manner, and physically unlock the plug.
This extremist minimalist design where we pretend things work magically when they don’t, just because it’s not aesthetic, is ridiculous.
It would take over 10 years for a hardware change to USB to be common. A 3rd party invention of a security plate eject key thingy would work. It could lock the device in place until power is gone from that socket.
With USB 4 just released, not going to happen via the USB people however.
No, its simple! Just put a capacitive touch sensor on the usb thumb drive. Any detected touch, would initiate the shutdown. No usb redesign necissary. No OS dependent functionality. Modest cost increase per drive.
Bill Shooter or Bul,
No, accidental touches would become annoying.
A physical eject button and a physical lock is the only empirically proven way. And you do need OS dependent functionality because that physical lock would need a signal from the OS to let it know when its writes are finished.
Would I trust Windows after decades of properly ejecting USB devices? Not a chance….