USB flash drive has bad superblocks. Sandisk said damage "likely caused by partitioning"

bkaral

Ars Tribunus Militum
2,646
Hi everyone:

I have an Ultra Fit 3.1 Gen1 USBFD that I use for storing logs and software for my (Linux-based) router. (I have very very little knowledge of Linux). It barely gets any activity, since logging is such a low data activity. However, a recent test done via the router output the following:

root@router:/tmp/home/root# fsck.ext4 /dev/sda1
e2fsck 1.47.0 (5-Feb-2023)
ext2fs_open2: Bad magic number in super-block
fsck.ext4: Superblock invalid, trying backup blocks...
fsck.ext4: Bad magic number in super-block while trying to open /dev/sda1

The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem. If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
or
e2fsck -b 32768 <device>




Fdisk showed the following partition data on the flash drive before I ran the test:

root@router:/tmp/home/root# fdisk /dev/sda

The number of cylinders for this disk is set to 7480.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p
Disk /dev/sda: 57 GB, 61530439680 bytes, 120176640 sectors
7480 cylinders, 255 heads, 63 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Device Boot StartCHS EndCHS StartLBA EndLBA Sectors Size Id Type
/dev/sda1 0,1,1 1023,254,63 63 58368062 58368000 27.8G 83 Linux
/dev/sda2 1023,254,63 1023,254,63 58368063 120176639 61808577 29.4G 83 Linux


I spoke with a rep. at Sandisk, and he claimed that this was probably caused by my creating more than one partition on the drive. Their drives, he said, "aren't made to handle more than on partition". That sounded like bullsh*t to me. I asked him for documentation as proof of his claim. So far, no proof.

He told me to repartition it with one partition only, and reformat. I did that. I'm running h2testw on it right now.

Can partitioning a Sandisk flash drive cause bad blocks, as Sandisk claims?
 
Last edited:

Lord Evermore

Ars Scholae Palatinae
1,490
Subscriptor++
That's bullshit. Partitioning is nothing but data on the drive that says "these blocks/sectors are one partition, these are another partition" so the OS doesn't have to track it and it can be connected to other machines. There is nothing about partitioning that can physically damage a block on any type of drive. However, it is possible to damage the partition table which could maybe cause this "bad magic number", rather than it necessarily being physical damage. But if it was good all this time, and now suddenly says it's corrupt, and is very specific about it, then there is a good chance that the block has physically failed. Repartitioning and scanning it for bad blocks could possibly allow them to be marked as bad so that they're not used, or allow it to re-allocate from spare. Since drives are designed with spares for this purpose, they probably wouldn't RMA it if it can be cleaned up that way. Flash drives of this type of course don't have the same automatic mechanisms like an HDD or SSD such as SMART that would just reallocate it and record the fact that it happened.
 
  • Like
Reactions: AndrewZ

bkaral

Ars Tribunus Militum
2,646
Thanks. That's exactly what I figured. They did offer a replacement IF my drive failed a test within Windows. Unfortunately (fortunately?) it just passed in h2test2w. Should I try to get it replaced anyways? I count on getting my router logs when I need to do troubleshooting.

If they won't replace it, would you still use this drive?

And would you please remind me...under what conditions do bad sectors on a USBFD such as this get reallocated?
 
Last edited:

Lord Evermore

Ars Scholae Palatinae
1,490
Subscriptor++
On an HDD or SSD the drive would ideally block bad sectors from use and mark the addresses to point to spare sectors automatically when it reaches a certain failure point, but it's honestly never seemed very reliable. That's just what it's SUPPOSED to do. I ran into far too many mechanical drives that were barely functional because they were having to re-read sectors multiple times, dragging the system performance to a crawl, with system files corrupted to the point that cloning the drive didn't even give a bootable OS. And SSDs often just die once some blocks become unreadable, but they do seem better at reallocating. A scan using chkdsk /b is supposed to reallocate blocks but also doesn't seem great at it. Ideally, if the drive physically reallocated a sector, a filesystem check shouldn't even see it. I think chkdsk's and other OS tools' bad sector checking and repair may be a leftover from before drives did that. That means only software looking at the SMART data will actually know, and flash drives don't have SMART.

I don't know what "test" Samsung expects you to run in Windows or any OS, since there is no manufacturer software for USB flash drives that they can say "this is the official test", and those companies often will just refuse to accept results from other software. There are tools like Flash Drive Tester from vconsole.com, and Steve Gibson at grc.com created ValiDrive (his website is so dated, and instead of a clear download link, you have to figure out that the screenshot of the program IS the download link).

Once a standard USB flash drive starts to go bad in any way, I usually trash it, but they almost always just become unusable by themselves on like the 3rd insertion after the initial signs of problems. They just don't cost enough to be worth trying to fix or RMA, and I never put anything irreplaceable on them anyway. If a replacement was going to cost like $30 or $40 then I'd try to RMA, but for 10 or 15, nah. If I had a drive that seemed to be working, but just seemed to have some corruption like in your case, I'd at least try a full reformat (not a quick one) and see what those test tools said, but if there are signs of actual functionality going bad, toss it.

What was the reason for running the test in the first place?
 

Paladin

Ars Legatus Legionis
32,552
Subscriptor
Best thing with that kind of situation is to treat them as basically disposable. Once it starts to have issues, you try a reformat or reinstall. If it fails, test it with whatever tool the manufacturer provides or lists on their support and then have them replace it if it is under warranty. No point in discussing its use with them, it only provides them a chance to deny warranty.

That said, it probably is fine. Many/most software makers will likely recommend not using portable flash devices like a USB thumb drive for an OS install. VMWare, TrueNAS, etc. have all changed from that old recommendation because the drives are just not meant to be reliable over long term use with frequent I/O. A normal SSD drive for an OS install gets near constant writes and reads all the time, even with a mostly idle system. A USB thumbdrive is usually used a couple of times a day or sometimes a few times a week or month in some cases. Orders of magnetude difference in use. Even with an application like a router log drive, it may still get thousands of writes and reads a day, depending on the logging settings (and if you turn them down low enough, why bother with logs at all?). If the operating system is designed with more reliable storage in mind, it is understandable that a flash thumbdrive won't do a great job over the long term.

Basically, I'm not saying don't use it like you are, I'm saying don't be surprised if it doesn't last as long as you might prefer and treat it like it was just being used for moving around files in Windows when it died or faulted. Heck, try formatting it with exfat and then copying some random files to it as a final test before you warranty replace it. If it can't do that, then it's ripe for replacement for sure.
 
  • Like
Reactions: Lord Evermore

Lord Evermore

Ars Scholae Palatinae
1,490
Subscriptor++
RAID10 the sons of bitches!

Dammit, I just gave myself a fun little project. I wonder how well a bunch of quality flash drives would do in RAID when connected to a hub versus each one connected directly to the motherboard on separate ports and separate chipset root hubs. It's been done before but I don't know about recently and with the newest (non-NVME) flash drives and a 3.2-rated hub and a bunch of drives.
 
Last edited:

cogwheel

Ars Tribunus Angusticlavius
6,691
Subscriptor
That's bullshit. Partitioning is nothing but data on the drive that says "these blocks/sectors are one partition, these are another partition" so the OS doesn't have to track it and it can be connected to other machines. There is nothing about partitioning that can physically damage a block on any type of drive.
Not necessarily. We're talking about thumb drives here, not SSDs. Thumb drives generally don't have as good wear leveling routines as SSDs, and @bkaral is using the thumb drive for about as bad of a use of flash as you can do (strongly write-dominated, repeated rewriting of the same files), so it's plausible that when you partition the thumb drive model in question, wear leveling just barfs (it can't understand the multiple file systems to know how to wear level) and the usage wore out one or more blocks of flash due to the repeated writes to those blocks.

You're correct, though, that the act of partitioning itself can't damage the flash.
 

Andrewcw

Ars Legatus Legionis
18,129
Subscriptor
Wait you're telling me other then proof of purchase from an authorized dealer the tech went through that much trouble shooting with you? The tried and true method of getting the RMA is. I can't get it to work on my system.

While logging is a low activity thing. A 1 byte change every second will cause excessive wear rather quickly on something with no wear leveling and even if it did have wear leveling. You'd be writting a whole swath of memory just for that 1 change. PfSense at one point had a warning never to use any flash based drive to record logs until they fixed their log writing routine.
 

m0nckywrench

Ars Tribunus Angusticlavius
6,852
I discovered a magical RMA tool many years ago which leaves no trace and ensures no other customer is frustrated by buggered hardware inflicted on them.

I made a piezo spark plug tester from a cheap piezo gas grill igniter, a spark plug boot and a piece of solid copper wire and discovered the high voltage low amp output leaves no burn marks while ensuring intermittent failures become permanent. Works a treat but for pure RMA use all one needs is the igniter, some wire and a bit of electrical or other tape to secure that wire to the ground of the grill igniter. I touch the output to one contact, the ground wire then cycle the igniter button a few times. Test to confirm item bricked and done.

Dead hardware is easier to RMA than intermittent hardware so my hardware is never intemittent. No one's time is wasted and I get my replacement. That said decent hardware doesn't fail often and I've only used it thrice in over two decades for anything but spark plugs and a potato cannon.
 
  • Like
Reactions: Danger Mouse

Lord Evermore

Ars Scholae Palatinae
1,490
Subscriptor++
I discovered a magical RMA tool many years ago which leaves no trace and ensures no other customer is frustrated by buggered hardware inflicted on them.

I made a piezo spark plug tester from a cheap piezo gas grill igniter, a spark plug boot and a piece of solid copper wire and discovered the high voltage low amp output leaves no burn marks while ensuring intermittent failures become permanent. Works a treat but for pure RMA use all one needs is the igniter, some wire and a bit of electrical or other tape to secure that wire to the ground of the grill igniter. I touch the output to one contact, the ground wire then cycle the igniter button a few times. Test to confirm item bricked and done.

Dead hardware is easier to RMA than intermittent hardware so my hardware is never intemittent. No one's time is wasted and I get my replacement. That said decent hardware doesn't fail often and I've only used it thrice in over two decades for anything but spark plugs and a potato cannon.
Unfortunately I have a conscience.
 

w00key

Ars Praefectus
5,907
Subscriptor
Unfortunately I have a conscience.
It's not like half working hardware is good for anything but recycling. It's the agents / AI chatbot's fault making you jump though these hoops.

But first make sure it isn't a software thing, unclean shutdown could lead to issues.

@bkaral most USB drives aren't meant for use as a disk 24/7. Buy an USB SSD next time, those have a proper controller with wear leveling.
 

bkaral

Ars Tribunus Militum
2,646
I discovered a magical RMA tool many years ago which leaves no trace and ensures no other customer is frustrated by buggered hardware inflicted on them.

I made a piezo spark plug tester from a cheap piezo gas grill igniter, a spark plug boot and a piece of solid copper wire and discovered the high voltage low amp output leaves no burn marks while ensuring intermittent failures become permanent. Works a treat but for pure RMA use all one needs is the igniter, some wire and a bit of electrical or other tape to secure that wire to the ground of the grill igniter. I touch the output to one contact, the ground wire then cycle the igniter button a few times. Test to confirm item bricked and done.

Dead hardware is easier to RMA than intermittent hardware so my hardware is never intemittent. No one's time is wasted and I get my replacement. That said decent hardware doesn't fail often and I've only used it thrice in over two decades for anything but spark plugs and a potato cannon.

You're serious about this? If so, can you explain more about how you set this up? I have one of those indoor butane stoves like they use in the Asian restaurants. I'm thinking the piezo igniter on that would work?
 

Lord Evermore

Ars Scholae Palatinae
1,490
Subscriptor++
Sandisk is RMAing the drive for me. However, they don't cover my shipping cost to them.

For for 24/7 logging, in this router, it logs maybe a few KB a day, so I don't think it's going to wear out the drive any time soon.
How much did you pay for this drive?

You're serious about this? If so, can you explain more about how you set this up?
Please do it in private.
 

Lord Evermore

Ars Scholae Palatinae
1,490
Subscriptor++
If they make you pay shipping, kind of seems like a waste of time/money. A new 16GB drive is less than $10 I would guess. Yeah, some less than $7 for the really cheap ones.
A single 64GB Ultra Fit (seems to be what OP has) is only $12.35 on Amazon, and a two-pack is $20.17. Hardly seems worth the effort and cost of packaging and shipping. And they'll probably send back a refurbished one.
 
Sandisk is RMAing the drive for me. However, they don't cover my shipping cost to them.

For for 24/7 logging, in this router, it logs maybe a few KB a day, so I don't think it's going to wear out the drive any time soon.
In this case it's not the volume of writes, it's the number of writes. If the router logs one line of text every minute that's a very low volume, but you're going to get incredible write amplification. Inexpensive thumb drives aren't robust in this kind of usage.
 

Paladin

Ars Legatus Legionis
32,552
Subscriptor
Eh, in that case, I would just get whatever cheap one you want and burn it out and throw it away and replace it. The time spent on RMA and the money spent on shipping is more value than another $5-10 flash drive. Heck, you can get Samsung Bar ones on Ebay for around $5.50 each (32GB), assuming they are legit. Even if they are not legit they might still work well enough for a good while. I would just buy 3 or 5 of them and keep them on hand for replacement.

Well... to be honest, I wouldn't save logs from a consumer router in the first place. There's no data there that is of any interest to me so... yeah. ;)
 

cogwheel

Ars Tribunus Angusticlavius
6,691
Subscriptor
It doesn't even log that much. It logs maybe a few lines every hour.
You're still assuming this thumbdrive is like a SSD. You can't assume that; the wear leveling may be much, much less advanced, such that repeated writes to the same file end up wearing out a single block instead of just moving the used block around all the free space on the drive.

Consider a well-regarded SSD, like the SK Hynix Gold P31. The 2TB P31 has a write endurance for the drive of 1,200 TB written. If we assume those writes are evenly distributed among the drive's flash blocks, that means each block is rated for 600 writes. At one write to a block per hour (i.e. wear leveling not working, which may be happening to your thumbdrive as I described in my prior post in the thread), you'll hit 600 writes in under a month! Even if we assume that the blocks can take significantly more than 600 writes, there are 8760 hours in a year, so if the wear leveling is absent or not working, you will wear out the flash.

This is the reason that it is not recommended to store logs on flash, unless the flash has working wear leveling for your exact usage and is drastically overprovisioned, or the logging system is flash-aware (e.g. logs are actually written to a ramdisk, which is synced to flash at a much lower frequency than the logs are updated).

And it's not like I have a lot of choice. This a consumer-grade router, so it's either this or a JFFS partition and this has worked better for me.
You do have a choice. You can use a USB SSD like the Samsung T series. Those are essentially a standard Samsung SSD sitting behind a USB bridge, complete with the wear leveling of a Samsung SSD.

You probably don't want to take this choice (the 500GB T7 costs ~$80, and USB SSDs from other reputable brands are similarly priced), but that doesn't mean the choice doesn't exist.
 

Paladin

Ars Legatus Legionis
32,552
Subscriptor
Assuming you are ok with the physical appearance and delicacy:


View: https://www.amazon.com/ELUTENG-M-2-NVME-NGFF-Converter/dp/B0BJCYR1P7



View: https://www.amazon.com/Hynix-BC901-256GB-Internal-Factor/dp/B0C26NP37B


Those together are under $30 (before tax) for a 256GB SSD on a USB port. Should work great.

Personally, I bet you are still fine with a $10 USB flash drive from Samsung or SanDisk. I've asked their support and both claimed (and have/had on their support websites) that they do wear leveling on their USB flash drives to prolong their life so that should not be a huge concern, though who knows if they really do it... Anyway, if a $10 one lasts 5 years or 3 years or whatever, it's probably fine.
 

Lord Evermore

Ars Scholae Palatinae
1,490
Subscriptor++
Assuming you are ok with the physical appearance and delicacy:





Those together are under $30 (before tax) for a 256GB SSD on a USB port. Should work great.

Personally, I bet you are still fine with a $10 USB flash drive from Samsung or SanDisk. I've asked their support and both claimed (and have/had on their support websites) that they do wear leveling on their USB flash drives to prolong their life so that should not be a huge concern, though who knows if they really do it... Anyway, if a $10 one lasts 5 years or 3 years or whatever, it's probably fine.

I'd say at least get an enclosure rather than a bare PCB, to protect the electronics. It's not that much more expensive (or not at all, with Prime). I think a cabled connector is also better as it doesn't require having it stick straight out of the device by 3 inches with no flexibility.


View: https://www.amazon.com/Yottamaster-Enclsoure-External-Enclosure-Heatsink/dp/B0BJ2JTT3W
(cheaper than the PCB-only too)

256GB and NVME is a ridiculous amount of storage and speed for this purpose, but we're at the point that anything going smaller becomes lower space/performance per dollar value. But, to pinch those pennies and still have more than needed...


View: https://www.amazon.com/Patriot-Internal-Solid-State-P210S128G25/dp/B08B3JBDJ6


View: https://www.amazon.com/GiGimundo-Enclosure-External-2-5inch-Supports/dp/B0BPXMZK6G
(or https://www.amazon.com/PHIXERO-Enclosure-Tool-Free-Supported-Transparent/dp/B0BNLD8Q43 if the deal on the first expires, or you don't want pink)
 
  • Like
Reactions: Paladin

cogwheel

Ars Tribunus Angusticlavius
6,691
Subscriptor
Yeah, I was looking for the smallest option to sort of simulate a thumbdrive but those are a good solution too. Just be careful to watch for NVME vs. SATA SSDs and enclosures. I've messed that up before.
yeah, like this:
I'd say at least get an enclosure rather than a bare PCB, to protect the electronics. It's not that much more expensive (or not at all, with Prime). I think a cabled connector is also better as it doesn't require having it stick straight out of the device by 3 inches with no flexibility.


View: https://www.amazon.com/Yottamaster-Enclsoure-External-Enclosure-Heatsink/dp/B0BJ2JTT3W
(cheaper than the PCB-only too)

Lord Evermore, this enclosure is SATA, and won't work with the SK Hynix m.2 drive Paladin linked.

But yeah, roll your own can be cheaper. I'd probably go for this one to get a proper enclosure and cabled connection.
 

Lord Evermore

Ars Scholae Palatinae
1,490
Subscriptor++
yeah, like this:

Lord Evermore, this enclosure is SATA, and won't work with the SK Hynix m.2 drive Paladin linked.

But yeah, roll your own can be cheaper. I'd probably go for this one to get a proper enclosure and cabled connection.
Shit, yeah, I had a different one to link originally. Includes a cable with a USB-C to A adapter so it could be used for other purposes if needed, and it's cheaper than the Orico and tool-free. I doubt there's going to be any need for cooling fins in this case with just occasional small writes. (I have a similar tool-free enclosure, and the contact is not as solid as one that is completely flat and closed with a screw since it has to be able to slide into the housing, it's spring compressed and at an angle, but it works well enough even with sustained activity, and even a throttled NVME SSD is faster than USB.)

View: https://www.amazon.com/MAIWO-NVMe-Enclosure-External-Capacity/dp/B0BF9CFMCC
 

cogwheel

Ars Tribunus Angusticlavius
6,691
Subscriptor
Includes a cable with a USB-C to A adapter so it could be used for other purposes if needed, and it's cheaper than the Orico and tool-free. I doubt there's going to be any need for cooling fins in this case with just occasional small writes.
I picked the Orico since that name (company?) predates the flooding of Amazon with "brands" consisting of 5-7 random capital letters like "MAIWO" or "ELUTENG", the Orico is only slightly more expensive than those "brands", and I have a few items from Orico that haven't failed yet. The fins are just aesthetics for me.
 

Lord Evermore

Ars Scholae Palatinae
1,490
Subscriptor++
I picked the Orico since that name (company?) predates the flooding of Amazon with "brands" consisting of 5-7 random capital letters like "MAIWO" or "ELUTENG", the Orico is only slightly more expensive than those "brands", and I have a few items from Orico that haven't failed yet. The fins are just aesthetics for me.
ELUTENG seems to be good enough, and at this end of the scale, with this type of device, it doesn't really seem like there's a huge difference. They're fundamentally the exact same device, probably made on the same production lines. (Though I will admit one Eluteng enclosure I got is flaky, but another is not.) This is the first time I've ever heard the name Orico, so it's not like it's a "name brand", either, and I wouldn't give it any more trust than I would Eluteng or Maiwo. And those at least seem like names, not just random letters. The fins are likely 20% of the additional price.
 

Paladin

Ars Legatus Legionis
32,552
Subscriptor
I've decided for now to stick with the RMA replacement Sandisk is sending me. I also bought a spare, for when that dies. I'll also backup my logs to a local PC just in case. Thanks everyone.
What kind of logs do you even get? I would expect it just logs when you access the router to make a configuration change. Everything from outside the router is likely blocked and you should not have the admin interface open to the internet so it should be basically just logs of you administrating the router.
 

Andrewcw

Ars Legatus Legionis
18,129
Subscriptor
I picked the Orico since that name (company?) predates the flooding of Amazon with "brands" consisting of 5-7 random capital letters like "MAIWO" or "ELUTENG", the Orico is only slightly more expensive than those "brands", and I have a few items from Orico that haven't failed yet. The fins are just aesthetics for me.
Yeah Orico pre-dates the flood. They built up a brand name as they sold as retail products on other sites. I've actually dealt with the RMA process for them for a power strip. I consider them on the high end of rebadging or even having some design influences of their own. And they've even go so far and send products to be tested for certifications when Qualcomm had a certification requirement to use their badging on products.
 
Did you build your own router using Linux or did you use a distro meant to be a router?

I use OpenWRT and it puts apps and configuration on flash, and dynamic data like logs and temp files on a ram drive. I then use a cron job to backup logs to a USB drive every hour, but I rename the file every day, and do not delete old logs, to avoid writing to the same flash and wearing it out.
 

Paladin

Ars Legatus Legionis
32,552
Subscriptor
What I consider the best practice for that kind of thing is to just use syslog. If you have any kind of server or computer that runs 24/7, you can easily use a syslog collector to accept logs from all kinds of things and then either rotate and expire them on a schedule or index them for search, and alerts, or whatever you want.
 
Yeah, syslog is the way to go if you have a 24/7 server.

Another solution is use an ancient USB drive. I have some old 4gb drives given out at trade shows that use MLC flash. They are slow, but can handle a lot of writes.

You can use this program to query the type of flash on a USB drive:
 
  • Like
Reactions: continuum

Lord Evermore

Ars Scholae Palatinae
1,490
Subscriptor++
Didn't the old Sandisk Cruzer Extreme (Pro?) use then-impressive SSD controllers and support more advanced wear leveling? But those were like 10 years ago.
SanDisk just keeps recycling the names for their devices, with slight variations like putting USB 3.0 in it, so it's hard to ever find anything specific by searching the name. SanDisk Extreme applies to a new thing every year, from flash drives to SD cards. It looks like it was just SanDisk Extreme but it did have impressive numbers. It took a while to find a review that said anything about the technology but they took it apart and did see a USB3.0 to SATA controller.
 

Wheels Of Confusion

Ars Legatus Legionis
66,177
Subscriptor
SanDisk just keeps recycling the names for their devices, with slight variations like putting USB 3.0 in it, so it's hard to ever find anything specific by searching the name. SanDisk Extreme applies to a new thing every year, from flash drives to SD cards. It looks like it was just SanDisk Extreme but it did have impressive numbers. It took a while to find a review that said anything about the technology but they took it apart and did see a USB3.0 to SATA controller.
Specifically I'm talking about a USB 3.0 thumb drive, not an SSD or memory card. And a rather hefty one, full-sized rather than miniature.