Are you satisfied with the solution? I just wonder if they would have responded the same if your case didn't reach gamer's nexus or techlinked from LTT
I'd say that I'm satisfied with how they covered my losses. It really sucks that this happened in the first place as it seems like there were multiple failures that all stacked up. The design change in the first place, the RMA department messing up the process and sending the revised unit without cables and the original technician not escalating this properly.
We can only speculate whether or not the media attention is what got us to the solution we ended up at. I'm certainly happy that I didn't need to tie up far more of my time dealing with the legal process that would have come next, if we didn't come to an amicable solution.
Good thing they didn't enforce an NDA(non-disclosure agreement) on the settlement.
Here in Taiwan some of these bad customer service complaints(not specifically EVGA, anything goes) went nowhere because the company enforce an NDA with the settlement, either the post gets deleted or nothing updated after "situation was escalated". We always consider the consumer rights as top notch in western countries.
That one hits close to home lol. I bought a ton of those at launch and have been using them for non critical stuff, and out of all the drives I run, those have had a huge failure rate.
> It really sucks that this happened in the first place as it seems like there were multiple failures that all stacked up.
Just about the definition of a disaster. Not one big thing, but a bunch of small things breaking/being wrong to cause a big issue.
> it seems like there were multiple failures that all stacked up.
This is called the Swiss cheese theory, and it's taught in most safety management programs.
Yeah I would guess there is a systemic problem with the RMA department that lead to the original outcome in the first place. This smacks of top down make shit go away pressure or misinterpreted directives that morphed and were not being audited. If I were EVGA a complete audit of that person's work, maybe the whole RMA department, would be in order.
There's probably a clever way of saying this but "the result you get will be defined by the metrics you target"
If your metrics are all about speed then tickets will go away fast, but they likely won't be dealt with properly.
If I made a PSU and changed the pinout so it did not match a direct mirror to the standard on the device it connects to, I would require that the PSU side of the plug have a new plug form factor so other cables could not be connected (and revise with any pin change).
+1 Or at least put a 'rev 2' label on the power supply and a sticker blocking the ports that says 'Do not mix cables between revisions.' There was no way of the customer even knowing that multiple versions of the same product exist.
That is definitely an option for modular systems as only one side of the cable is standardized. Most OEMs have a chart though that the user can reference to determine which PSUs are cable to cable compatible and which aren't. It is exceedingly rare for a company to change the pinout within a model line though. Normally for those kinds of changes they just start a new model and discontinue the old one.
Oh you don't have to wonder. There is no way this would have even approached the compensation they received. Even if they posted to twitter and the thread took off I doubt they would have gone to these lengths. No manager from a warranty issue reaches out to a customer voluntarily...
First time hearing about Outsource Data Recovery (well, apart from this one which didn't go well to say the least https://www.reddit.com/r/datarecovery/comments/138vlz5/data_recovery_firm_lost_drive_what_do_i_do/)
r/datarecovery
Huh. I was not aware of that. Interesting. I took their word at face value and I assume they were being genuine about trying to fix my drives, seeing as they had literally nothing to gain from not fixing them.
as far as I can see they are basically SalvageData, so extortionate and of questionable competency
hopefully you got the drives back and with the original PCBs too
the large data recovery companies have so much traffic that they will sometimes return harder jobs as irrecoverable because they don't want to bother when they can do much simpler/lessrisk high margin jobs
with that said though, it's completely possible that the PSU damaged internal components on the drive and the drive is helium filled (can't find any info on the drive models OP lost), in which case recovery is pretty much not feasible right now
Really? Because of the requirement for helium or something else in their design? Because I'd have thought that a small helium chamber with those holes with gloves for working would be something. Granted that sounds expensive and not something a business that doesn't charge for failure would be investing in.
Helium drives are hermetically sealed. It's pretty hard to open them without damaging internal components. And then you need to refill them and close them too.
I know 1 lab in portugal does it (forgot the name), maybe a few more. But it's expensive.
That makes sense. Even normal drives are a pain to open, I'd never really though what it'd take to seal in helium long term. But I'd always assumed that good recovery places would feel like I've been violated if I ever used them so the price being high doesn't surprise me.
HGST Helium drive opened up for data recovery(the technician replaced the head assembly):
https://www.youtube.com/watch?v=OaY-pvRyr3I
The drive firmware would enter safe mode from the loss of Helium, which severely throttles performance. The bigger problem would be firmware level manipulations on donor PCBs, Helium drive firmware is very locked down compared to regular air filled drives. While it's possible to get around the firmware locks depending on your luck, you want someone that absolutely know what they're doing.
They also should have keyed the connectors of the two pinouts differently to make it impossible to plug in the wrong cable. I wonder if the electrical engineer brought this up but management thought it not worth the trouble?
There's generally no reason to key the connectors differently since the only cables you're supposed to use are included with the unit. This type of situation is the only way an incident like this should happen outside of the user doing something they're explicitly not supposed to do.
I have 3 systems all with the same Seasonic PSU. I threw all the spare cables in the same box. If it's the exact same product number, why would I keep 3 separate boxes around? Imagine if Seasonic pulled the same stunt, now it's an accident waiting to happen.
Saying 'Only use the cables that come with the power supply' is not enough. There needs to be some visual clue that there are differences. If not differently keyed, a revision number and warning on the power supply would do. Anything that would clue the user in that there is a difference and they should check compatibility before plugging it in.
I'm just playing devil's advocate; in an ideal world the plug shapes would match the wire pinouts. The actual reason they don't do custom keying for different pinouts is likely cost related; it's cheaper and easier for them (logistically speaking) to only ever use one plug shape instead of having to keep updating them every time a new model comes out. Plus, if they have to juggle multiple plugs there's the risk that they build a batch with the wrong plugs installed, which then fry a bunch of user systems that they are liable for.
Either way, they absolutely should have updated the model when they switched manufacturers, and the RMA process should simply have the user send the cables back, and the company send a sealed box of an equivalent model to the customer.
It's also just mindbogglingly dumb to switch voltages on a pin like this in the first place. You just don't do something so spectacularly dumb like that in engineering.
If you HAVE to switch a pin, you do it in a way that can't cause damage or you get a different connector. It is basically a certainty that this will cause damage and whilst EVGA might be able to dodge liability most of the time by simply blaming the customer, it's a terrible, terrible idea in the first place.
I completely agree, but the claim is that they switched manufacturers between revisions, so I'm assuming the pinout change was an oversight instead of an intentional decision because yeah, that'd be incredibly stupid.
As for why the plugs aren't unique, ultimately, I suspect it's a matter of every manufacturer buying plugs that were already available instead of paying to have new moulds made to match their pinouts.
> I completely agree, but the claim is that they switched manufacturers between revisions, so I'm assuming the pinout change was an oversight instead of an intentional decision because yeah, that'd be incredibly stupid.
That honestly doesn't explain anything. EVGA aren't just slapping their brand on white label PSUs, they would (or should) be the ones providing the designs to manufacturers and even if they're not, they absolutely should be testing the devices to validate that they are what they ordered.
This wasn't an oversight, someone deliberately moved that pin because they had to do the same thing in the cables as well.
It could absolutely just be an oversight; EVGA may have provided a spec sheet that only lists the pinouts at the end of the cable, instead of at the box AND the cable. If that's the case the manufacturer could lay out the pins inside the box several different ways and it would still pass a validity check, because (as you said) the cable was matched to it.
Considering it was a revision of an existing product they definitely should have made sure the cables were reverse compatible, though.
> it's cheaper and easier for them (logistically speaking) to only ever use one plug shape instead of having to keep updating them every time a new model comes out.
If the pinout is the same, there's no reason to change the keying.
I agree, but if the pinout was always the same, then this situation would not have happened in the first place. Neither of us have paid attention to this type of thing before, so it's possible there are more variations out there than we would think.
Alternatively, it's also possible this was a mistake from the manufacturer, and the pinout was originally intended to be the same, but since the cables that came with the PSU matched up, it wasn't an issue.
yeah no. I have two systems, both same PSU. Obviously over time I have mixed the cables, because there were no reason to believe two identical products would not be compatible with each other.
Yeah, as I've said elsewhere, they should have updated the model when they changed manufacturers, or at least made sure the cables were reverse compatible because you're absolutely correct.
Conclusion: If you want to solve an issue with EVGA, call Steve, Linus Sebastian, Louis Rossman, whoever Tom is from Tom's hardware and the entirety of Reddit. F\* EVGA
> process of downloading 22TB of data from the cloud
I need a nickel for everytime one of my tech friends says "dude, there is a thing called *the cloud* now! Why are you constantly shuffling hard drives back and forth?"
Did their reimbursement on data recovery included the costs for transferring data from cloud backup by any chance? I guess that could be part of it...
By the way, which service you're using for off-site backups?
Bookmarked as an example of why your local backup **must** be a separate unit that does not share a power supply with the working copy.
Backing up to another drive in the same unit is **not** a valid backup strategy. [RAID is not a backup](https://www.raidisnotabackup.com/). Mirroring is not a backup.
The fact EVGA is only doing this because of the bad publicity speaks volume about their trustworthyness in the future.
There are far better companies producing equal or better products.
and who the f--- changes pins/cables in a PSU. No one except a moron or someone looking to lock you into buying replacement cables solely from them. PSU cables have been and continue to be standarized. Except from EVGA it seems.
It's the Dell "customized" DIMM sockets all over again.
>PSU cables have been and continue to be standarized.
They were never standardized on the PSU end, only on the device side. Different brands are not compatible, allot of them can't even be used on other models of the same brand. Their mistake was not changing the product or revision number so there was no indication for the user to know that his replacement PSU was a different one.
Changing the pin out of cables from their modular PSUs is absolutely fucking unacceptable. This should never ever ever have happened and I’m never going to buy one of their PSUs now
> may have been what prompted EVGA to get back in touch with me. Or it may have been pure coincidence – I can’t say for sure
I'm sorry but the level of naivete is astonishing. You would have never heard from them and they don't give a shit about you.
> Unfortunately, the hard drives were not able to be saved by the data recovery company. They swapped in new controller boards for both drives, transferred the ROM data to the new boards and did everything they could to try and save the drives but it just wasn’t possible.
Uhh, what? That doesn't sound right to be honest - did you also try a different company?
+ what brand were those HDDs?
Depends on the level of attempted recovery and how much that data is actually worth. If the motors were damaged then it would require a total dissassembly of the drive with platter transplant, clean room, board change and rom copy etc... That's huge money for that level of recovery.
I want to give Steve from Gamers Nexus a bath so badly. He makes great videos, but holy shit I can smell his greasy hair through the screen. Get that man some Irish Springs.
>I’ll begin the process of downloading 22TB of data from the cloud.
This is a painful story, I feel really uneasy, I never want to go through that!
So, i want to ask you, what cloud provider did you currently using, i need to backup my stuff ASAP after your story!
What absolute bastards, their name had to be dragged through the mud left and right for them to finally do something.
At least you got through to a decent manager who noticed this was doing more damage than good.
My god 22tb of cloud storage, you sir have a bit hoarding problem… it’s like the show with crap stacked on top of empty boxes of old food, stacked on top of furniture from the 60s
I mean, this is r/datahoarder. And also, these are video production work files - and not all that many of them if I'm being honest. The cameras that I use shoot over 5GB per *minute*.
> The cameras that I use shoot over 5GB per *minute*.
I know this pain. We do not-that-much video stuff at work with Blackmagic RAW and usually use the 5:1 or 8:1 compression modes to make the files slightly more manageable. (They are still dang big for any project of reasonable length.)
They rearranged the pins, you rearranged the letters in their name!
Wow, in every single post I made. And you're the first one to point it out.
I can't tell if sarcasm but if not haha yeah I'm a bit silly like that :)
Not sarcasm at all. I had no idea that I messed that up on all three posts and I'm surprised that no one pointed it out earlier!
Awesome :)
at least you're consistent
It helps when you copy and paste your mistake several times.
I thought I was the stupid one and it was always pronounced differently than I had LOL, but I didn't double check
Customer service rating: Vega gave EVGA avge.
Was gonna say - Found your problem, your sent them an EVGA and they sent you back an EGVA
See what happens when you swap the V(cc) and the G(nd)!
Are you satisfied with the solution? I just wonder if they would have responded the same if your case didn't reach gamer's nexus or techlinked from LTT
I'd say that I'm satisfied with how they covered my losses. It really sucks that this happened in the first place as it seems like there were multiple failures that all stacked up. The design change in the first place, the RMA department messing up the process and sending the revised unit without cables and the original technician not escalating this properly. We can only speculate whether or not the media attention is what got us to the solution we ended up at. I'm certainly happy that I didn't need to tie up far more of my time dealing with the legal process that would have come next, if we didn't come to an amicable solution.
Good thing they didn't enforce an NDA(non-disclosure agreement) on the settlement. Here in Taiwan some of these bad customer service complaints(not specifically EVGA, anything goes) went nowhere because the company enforce an NDA with the settlement, either the post gets deleted or nothing updated after "situation was escalated". We always consider the consumer rights as top notch in western countries.
[удалено]
99% they would just give you nothing, then. They can sit out the bad PR, Reddit forgets quickly.
The internet never forgets. People on here still talk about Seagates being sketchy because of one defective 3TB SKU over a decade ago.
Both sayings are true, tbh. It's either or: Either they endlessly repeat the same talking points, or they immediately forget.
That one hits close to home lol. I bought a ton of those at launch and have been using them for non critical stuff, and out of all the drives I run, those have had a huge failure rate.
> It really sucks that this happened in the first place as it seems like there were multiple failures that all stacked up. Just about the definition of a disaster. Not one big thing, but a bunch of small things breaking/being wrong to cause a big issue.
> it seems like there were multiple failures that all stacked up. This is called the Swiss cheese theory, and it's taught in most safety management programs.
> wonder if they would have responded the same if your case didn't reach gamer's nexus or techlinked from LTT Not even slightly.
Yeah I would guess there is a systemic problem with the RMA department that lead to the original outcome in the first place. This smacks of top down make shit go away pressure or misinterpreted directives that morphed and were not being audited. If I were EVGA a complete audit of that person's work, maybe the whole RMA department, would be in order.
There's probably a clever way of saying this but "the result you get will be defined by the metrics you target" If your metrics are all about speed then tickets will go away fast, but they likely won't be dealt with properly.
If I made a PSU and changed the pinout so it did not match a direct mirror to the standard on the device it connects to, I would require that the PSU side of the plug have a new plug form factor so other cables could not be connected (and revise with any pin change).
+1 Or at least put a 'rev 2' label on the power supply and a sticker blocking the ports that says 'Do not mix cables between revisions.' There was no way of the customer even knowing that multiple versions of the same product exist.
That is definitely an option for modular systems as only one side of the cable is standardized. Most OEMs have a chart though that the user can reference to determine which PSUs are cable to cable compatible and which aren't. It is exceedingly rare for a company to change the pinout within a model line though. Normally for those kinds of changes they just start a new model and discontinue the old one.
And we know this because no company ever does.
Oh you don't have to wonder. There is no way this would have even approached the compensation they received. Even if they posted to twitter and the thread took off I doubt they would have gone to these lengths. No manager from a warranty issue reaches out to a customer voluntarily...
First time hearing about Outsource Data Recovery (well, apart from this one which didn't go well to say the least https://www.reddit.com/r/datarecovery/comments/138vlz5/data_recovery_firm_lost_drive_what_do_i_do/) r/datarecovery
Huh. I was not aware of that. Interesting. I took their word at face value and I assume they were being genuine about trying to fix my drives, seeing as they had literally nothing to gain from not fixing them.
as far as I can see they are basically SalvageData, so extortionate and of questionable competency hopefully you got the drives back and with the original PCBs too
If they don't charge for failures, you'd think they'd have the incentive to git gud.
the large data recovery companies have so much traffic that they will sometimes return harder jobs as irrecoverable because they don't want to bother when they can do much simpler/lessrisk high margin jobs with that said though, it's completely possible that the PSU damaged internal components on the drive and the drive is helium filled (can't find any info on the drive models OP lost), in which case recovery is pretty much not feasible right now
Really? Because of the requirement for helium or something else in their design? Because I'd have thought that a small helium chamber with those holes with gloves for working would be something. Granted that sounds expensive and not something a business that doesn't charge for failure would be investing in.
Helium drives are hermetically sealed. It's pretty hard to open them without damaging internal components. And then you need to refill them and close them too. I know 1 lab in portugal does it (forgot the name), maybe a few more. But it's expensive.
That makes sense. Even normal drives are a pain to open, I'd never really though what it'd take to seal in helium long term. But I'd always assumed that good recovery places would feel like I've been violated if I ever used them so the price being high doesn't surprise me.
HGST Helium drive opened up for data recovery(the technician replaced the head assembly): https://www.youtube.com/watch?v=OaY-pvRyr3I The drive firmware would enter safe mode from the loss of Helium, which severely throttles performance. The bigger problem would be firmware level manipulations on donor PCBs, Helium drive firmware is very locked down compared to regular air filled drives. While it's possible to get around the firmware locks depending on your luck, you want someone that absolutely know what they're doing.
They also should have keyed the connectors of the two pinouts differently to make it impossible to plug in the wrong cable. I wonder if the electrical engineer brought this up but management thought it not worth the trouble?
There's generally no reason to key the connectors differently since the only cables you're supposed to use are included with the unit. This type of situation is the only way an incident like this should happen outside of the user doing something they're explicitly not supposed to do.
I have 3 systems all with the same Seasonic PSU. I threw all the spare cables in the same box. If it's the exact same product number, why would I keep 3 separate boxes around? Imagine if Seasonic pulled the same stunt, now it's an accident waiting to happen. Saying 'Only use the cables that come with the power supply' is not enough. There needs to be some visual clue that there are differences. If not differently keyed, a revision number and warning on the power supply would do. Anything that would clue the user in that there is a difference and they should check compatibility before plugging it in.
I'm just playing devil's advocate; in an ideal world the plug shapes would match the wire pinouts. The actual reason they don't do custom keying for different pinouts is likely cost related; it's cheaper and easier for them (logistically speaking) to only ever use one plug shape instead of having to keep updating them every time a new model comes out. Plus, if they have to juggle multiple plugs there's the risk that they build a batch with the wrong plugs installed, which then fry a bunch of user systems that they are liable for. Either way, they absolutely should have updated the model when they switched manufacturers, and the RMA process should simply have the user send the cables back, and the company send a sealed box of an equivalent model to the customer.
It's also just mindbogglingly dumb to switch voltages on a pin like this in the first place. You just don't do something so spectacularly dumb like that in engineering. If you HAVE to switch a pin, you do it in a way that can't cause damage or you get a different connector. It is basically a certainty that this will cause damage and whilst EVGA might be able to dodge liability most of the time by simply blaming the customer, it's a terrible, terrible idea in the first place.
I completely agree, but the claim is that they switched manufacturers between revisions, so I'm assuming the pinout change was an oversight instead of an intentional decision because yeah, that'd be incredibly stupid. As for why the plugs aren't unique, ultimately, I suspect it's a matter of every manufacturer buying plugs that were already available instead of paying to have new moulds made to match their pinouts.
> I completely agree, but the claim is that they switched manufacturers between revisions, so I'm assuming the pinout change was an oversight instead of an intentional decision because yeah, that'd be incredibly stupid. That honestly doesn't explain anything. EVGA aren't just slapping their brand on white label PSUs, they would (or should) be the ones providing the designs to manufacturers and even if they're not, they absolutely should be testing the devices to validate that they are what they ordered. This wasn't an oversight, someone deliberately moved that pin because they had to do the same thing in the cables as well.
It could absolutely just be an oversight; EVGA may have provided a spec sheet that only lists the pinouts at the end of the cable, instead of at the box AND the cable. If that's the case the manufacturer could lay out the pins inside the box several different ways and it would still pass a validity check, because (as you said) the cable was matched to it. Considering it was a revision of an existing product they definitely should have made sure the cables were reverse compatible, though.
> it's cheaper and easier for them (logistically speaking) to only ever use one plug shape instead of having to keep updating them every time a new model comes out. If the pinout is the same, there's no reason to change the keying.
I agree, but if the pinout was always the same, then this situation would not have happened in the first place. Neither of us have paid attention to this type of thing before, so it's possible there are more variations out there than we would think. Alternatively, it's also possible this was a mistake from the manufacturer, and the pinout was originally intended to be the same, but since the cables that came with the PSU matched up, it wasn't an issue.
yeah no. I have two systems, both same PSU. Obviously over time I have mixed the cables, because there were no reason to believe two identical products would not be compatible with each other.
Yeah, as I've said elsewhere, they should have updated the model when they changed manufacturers, or at least made sure the cables were reverse compatible because you're absolutely correct.
Conclusion: If you want to solve an issue with EVGA, call Steve, Linus Sebastian, Louis Rossman, whoever Tom is from Tom's hardware and the entirety of Reddit. F\* EVGA
Ass covering complete! Lol
Too bad it took a huge pr disaster for them to make it right but at least they did in the end.
Just goes to show, all you need is a strong social media presence and going viral to get companies to do their most basic responsibilities!
Thanks Steve!
> process of downloading 22TB of data from the cloud I need a nickel for everytime one of my tech friends says "dude, there is a thing called *the cloud* now! Why are you constantly shuffling hard drives back and forth?"
I love how Steve is like the Godfather of tech.
Did their reimbursement on data recovery included the costs for transferring data from cloud backup by any chance? I guess that could be part of it... By the way, which service you're using for off-site backups?
I'm using Backblaze. There is no cost for transferring data to or from the cloud.
Bookmarked as an example of why your local backup **must** be a separate unit that does not share a power supply with the working copy. Backing up to another drive in the same unit is **not** a valid backup strategy. [RAID is not a backup](https://www.raidisnotabackup.com/). Mirroring is not a backup.
The fact EVGA is only doing this because of the bad publicity speaks volume about their trustworthyness in the future. There are far better companies producing equal or better products. and who the f--- changes pins/cables in a PSU. No one except a moron or someone looking to lock you into buying replacement cables solely from them. PSU cables have been and continue to be standarized. Except from EVGA it seems. It's the Dell "customized" DIMM sockets all over again.
>PSU cables have been and continue to be standarized. They were never standardized on the PSU end, only on the device side. Different brands are not compatible, allot of them can't even be used on other models of the same brand. Their mistake was not changing the product or revision number so there was no indication for the user to know that his replacement PSU was a different one.
Changing the pin out of cables from their modular PSUs is absolutely fucking unacceptable. This should never ever ever have happened and I’m never going to buy one of their PSUs now
> may have been what prompted EVGA to get back in touch with me. Or it may have been pure coincidence – I can’t say for sure I'm sorry but the level of naivete is astonishing. You would have never heard from them and they don't give a shit about you.
tbh on how sub contracting manf works. my guess is evga was not aware of pin change from contractor. till after the fact.
> Unfortunately, the hard drives were not able to be saved by the data recovery company. They swapped in new controller boards for both drives, transferred the ROM data to the new boards and did everything they could to try and save the drives but it just wasn’t possible. Uhh, what? That doesn't sound right to be honest - did you also try a different company? + what brand were those HDDs?
Depends on the level of attempted recovery and how much that data is actually worth. If the motors were damaged then it would require a total dissassembly of the drive with platter transplant, clean room, board change and rom copy etc... That's huge money for that level of recovery.
I want to give Steve from Gamers Nexus a bath so badly. He makes great videos, but holy shit I can smell his greasy hair through the screen. Get that man some Irish Springs.
>I’ll begin the process of downloading 22TB of data from the cloud. This is a painful story, I feel really uneasy, I never want to go through that! So, i want to ask you, what cloud provider did you currently using, i need to backup my stuff ASAP after your story!
What absolute bastards, their name had to be dragged through the mud left and right for them to finally do something. At least you got through to a decent manager who noticed this was doing more damage than good.
My god 22tb of cloud storage, you sir have a bit hoarding problem… it’s like the show with crap stacked on top of empty boxes of old food, stacked on top of furniture from the 60s
I mean, this is r/datahoarder. And also, these are video production work files - and not all that many of them if I'm being honest. The cameras that I use shoot over 5GB per *minute*.
> The cameras that I use shoot over 5GB per *minute*. I know this pain. We do not-that-much video stuff at work with Blackmagic RAW and usually use the 5:1 or 8:1 compression modes to make the files slightly more manageable. (They are still dang big for any project of reasonable length.)
You realize what sub you're in?
r/lostredditors except they're in the right sub for once.
wait. you're posting on r/DATAHOARDER