Today we are going to share the result of a bit of investigation that started a few months ago on STH. The short version, it appears as though the Dell EMC S5200-ON series switches, the company’s high-end 25GbE-200GbE switches, have license/ royalty stickers that have a different company name on them than they should have. Instead of saying “American Megatrends”, they instead said “American Megatrands”. To give some perspective, this looks strange because it would be like buying a Dell notebook and getting a “Macrosoft Wandows” license sticker on it.
[…]Through a fairly rough October, we validated that indeed these stickers are in the wild. Ultimately, after we brought their existence to American Megatrends (AMI) and Dell’s attention (HPE did not care enough to investigate), we now have an artifact that says that American Megatrends is honoring the license stickers and will not pursue legal action against Dell’s customers or those using them.
This may seem like something insignificant and innocuous, but supply chain security is a big, big deal, and the fact these clearly misspelled license/royalty stickers made their way from printing down to the end-user of not just corporate hardware but supercomputers for the US military is… Concerning, to say the least. It shows that tampering with hardware anywhere between production of the individual chips and components down to delivery by the delivery person might be a lot easier to do than we think.
It seems to be common for the big name vendors to buy chips from suppliers who don’t have the security, etc. in place.
For example talk shows how things work in practice:
https://www.youtube.com/watch?v=ruEn7TE4YMM
You can buy rolls of PAT test labels on EBay. Admittedly it’s a niche use and the chance of a health and safety incident is low but a label doesn’t prove anything more than it’s a label. Most sites which might be touchy about such things probably have reasonable site security to match but it can take only one dodgy piece of equipment to kill people or cause millions of pounds worth of damage.
I bought some Duo lash glue off Ebay once. The first order was fine. Another order struck me as a bit odd and it turned out on reflection to be PVA glue. I had my doubts about it and visited Boots to buy some more and that was definitely the real thing and unmistakably gluey in a way PVA is not. The fake was some very expensive PVA glue! I’m much more touchy about supply chain issues after that.
PVA glue is actually fine for lashes according to a woman in Boots who said she used it when working abroad as an actress in New York. Originally made for stage actors working under hot lights some of the boutique fixing sprays with extra hold have a magic ingredient which they use to justify their three times the average inflated price. The magic ingredient is a percentage of PVA glue. That’s less than one pence worth.
Is this for real? This is what the tech bloggers are focusing on now? A random label with one wrong letter…. JFC.
javiercero1,
The the letters themselves are not the issue, however typos typically mean inauthentic parts, which could be a huge issue in otherwise authentic equipment used in secure contexts. Alas, since our normal supply chains run through china every day, it’s realistic that components get swapped without being noticed if they are subtle. Investigations will have to run their course to determine whether supply chains were compromised.
Or sometimes a single letter error on a sticker is just that… a single letter error on a sticker.
Hilarious to see how desperate for content that STH site is.
Yes, so laughable until your fake part fails at a critical point, be it a medical part keeping someone alive or a fake bolt in a plane that fails in mid-air.
And if you think it does not happen – IT ALREADY HAS.
The problems with the supply chains are driving the purchase agents to look elsewhere for parts, meanwhile there are scammers who are willing supply mislabeled parts to make a killing (pun intended) in profit and often the problems occur so much later it can not be traced to them.
It just show how lossy the QC has became.
Or the equipment doesn’t work at all.
Or a bunch of data gets exfiltrated ending up on BitTorrent.
Anyone else remember the infinite 32GB USB flash drives? They really only stored something like the last 512MB of data, but would look like they wrote everything by looping through the flash. That was pretty ingenious, and I have to give the scammers credit for that one.
Actually the problems can be easily traceable. I don’t think any of you have seen the contracts involved in these supply chains or how HW designs are brought up. Dell or whoever clearly knows where and from whom this part came from.
But most importantly, I don’t think many of you are aware of the economic barriers of entry involved. Making a counterfeit of this specific part would be not an economically viable scam in the least. It would take too much effort to make a counterfeit monitoring subsystem like this one, for such a small market and to jeopardize future orders.
Furthermore, all the article is showing is a typo and making some wild claims based on it. There’s no investigation about whether the firmware is legit or not. Or if there was a supply issue. Etc, etc.
It’s an article pandering to the actual supply chain issues.
@Jav
I had a purchasing issue a few months ago which triggered a complaint and refund. The feedback was they had sourced from a long-term trusted supplier. After investigation they said that someone had mixed the stock up with another line hence the iffy item.
As an extension to this you can also buy warehouses of surplus items from grade A on downwards and swap labels or make a firmware switch for little cost. The manufacture cost may be very high, the purchase price very low, and the sell-on price high enough to give a profitable margin. In fact there’s entire market segments especially in China based entirely off this model. They’re not necessarily fraudulent but mistakes can happen and some items may indeed be fraudulently passed off. A customer who doesn’t know what to look for may miss this and only discover the faults and off specification problems later. One day it may be a router. Another day a phone. Another day something else.
Taking a kneejerk unipolar view can cause you to miss the obvious and it’s not like this hasn’t been given enough publicity over the past year. It’s not even a new problem and existed before the internet became a thing. irrc, some counterfeit items are actually as good as or sometimes better than the original part, although the overwhelming majority are low quality fakes or items which failed original certification.
If people think the tech industry has a problem with this it’s nothing compared to the beauty or accessory industry or construction tools industry. Even the food industry has had supply chain issues with unsafe horse meat being passed off a beef for use in processed foods. Yes, you can buy Hermes or Louboutin knock offs which require close scrutiny to tell but I wouldn’t buy one. I don’t see the point.
javiercero1,
And yet counterfeit products and components do happen.
Can you quote the specific article claims you have a problem with so that everyone is on the same page?
I have a bunch of Intul X710 quad cards I’m selling for cheap. They’re the real deal and are totally not a scam, promise. 🙂
Let me know if you’re interested.
Cryptocurrency only, btw.
Flatland_Spider,
Do you take bitcon?
While it falls more into the consumer area it does touch on the area of “buyers” and quality of information. I’ve been looking into moisturisers for older women. Women tend to look after ourselves more as well as skin being thinner and drier so moisturiser is a thing. As you get older you begin to lose the collogen producing ability of the skin and the same is true for men, actually. Behind the “dark pattern” marketing and technobbble there are very few active ingredients. The rest are preservatives and emulsifiers and fillers and colouring. I usually buy E4. It’s a generic all purpose moisturiser which is really cheap but I’ve been looking into others. This is where it gets expensive.
Almost all over the counter moisturisers containing Ritalin have a huge markup. You can easily pay £100 for a small pot of “high end” moisturiser containing ritalin plus hydrating ingredients. Ritalin is actually a trademark and a family of chemicals of differing composition which target callogen production. More modern type two and now type three Ritalin variants are better for more sensitive skin or more effective. Without going into the gory details you can “titrate” from a low dose Ritalin to a higher dose Ritalin (under the guidance of a dermatologist if you want to) so you don’t suffer from short term cosmetic effects or side effects. After progressing from over the counter often price inflated products if you don’t chose wisely around a year to a year and a half depending on how you manage things you can progress to the most powerful form of Ritalin. This is a generic out of patent formula originally developed for acne. According to dermatologists you can mix it with a moisturiser or immediately apply it before or after a moisturiser so it’s easy to use and effective. Far from costing £100 a pot it costs £5 a tube which lasts for months and is ten times more powerful (hence the titration and consulting a dermatologist and usually prescription only nature of the product although you can buy it without one if you know where to buy it and know what you are doing). I’ll skip the details but anyone considering this product (not just women but men too) should look out for a night time moisturiser and use SP25 minimum preferably SP30 during the day.
I expect someone is going to moan “talk about the tech”. It is tech and yes it does cover “buying” as a function and supply chains. It’s just not a tech you are used to talking about. I’ve found the very best brand name outlets have very good buyers or if they don’t themselves an agency or consultants behind the scenes who do. It’s often an overlooked job but critical to merchandising and quality control and pricing. Even as an ordinary customer (I hate the term “consumer”) you are still a buyer with all the same challenges even if it may be at a more amateur level or without the latest information or ability to “walk the factory floor” professional buyers have. It doesn’t matter whether you’re talking graphics cards or anti-aging moisturisers. The lessons are all the same hence benchmarking (which is a formal business process) and looking at other industries and markets and seeing what you can learn.
Oh, and speaking of objections “dude” in the topic title is exclusionary language probably best avoided.
Did you accidentally post on the wrong place, or trolling as usual?
Why are you doing this? Why are moderators allowing you to do this?
I am glad this isn’t a case of counterfeit chips working their way in the supply chain, but in general, tech companies must eventually admit there are consequences to outsourcing so much production to a country like China that’s known for not caring much about counterfeiting and is also a potential adversary. Back in the old days, tech companies knew that all their chips and other components were made either in the US or in some well-established industrial zone in Japan. Not anymore.
For example, a common problem with outsourcing to China is “third-shift” manufacturing, where the person who owns the factory opens the factory during the night (after the first and second shifts are done) and cranks out more product to sell out of the back door, potentially using reject materials:
https://www.bunniestudios.com/blog/?page_id=1022
Or the factory just keeps producing the goods even after the license has expired:
https://money.cnn.com/magazines/fortune/fortune_archive/2006/05/01/8375455/index.htm (long read, but a good insight on how China’s courts handle such issues)
kurkosdr,
Yes exactly!
Counterfeit products are always unauthorized, but their source can vary from 3rd party clones to night runs that are virtually indistinguishable to a legitimate product outside of serial number anomalies. And there’s everything in between too. It could come from the same production line as official products but substituting cheaper components. I’ve found this to be common with arduinos, many of which are counterfeit that can be confirmed by markings.
It wouldn’t be fair to say chinese manufacturers are always guilty, but unless you can monitor the factory floor 24/7 it would be difficult to prove there aren’t counterfeit runs. There are inherent risks to outsourcing and this is one of them.
Edit:
Anyone remember when fake cpus made it into official newegg channels?
https://www.zdnet.com/article/newegg-sold-counterfeit-intel-core-i7-cpus/
Did you buy those Arduinos from an official store or random eBay and Amazon sellers?
For stuff still in production, I always buy from the official store(s) linked in the product website, even if the shipping options are not that great compared to-say-Amazon.
kurkosdr,
I think it was off ebay, I bought a lot of sets at different times so I don’t remember where all my stock comes from. I didn’t really care about it too much as long as they worked.
For me it depends on the product, if it’s something where I want a warranty, then I do that as well. I bought a projector direct from the manufacturer for this reason.
In another instance I bought some cat6e cables off amazon. However the cables turned out not to be to spec and amazon refunded the order. But I learned that it’s practically impossible to know what your getting since they can say whatever they want to make the sale. I tried again on newegg and got good cables. I limit my purchases on newegg to those “sold by newegg”. I don’t think “newegg marketplace” is any better than ebay/amazon.
There’s a pretty interesting video about counterfeit parts in the airline industry. The video is about an airline disaster in the 80s, but they get into counterfeit parts at 42:24.
https://www.youtube.com/watch?v=sq5Km1zes4o
The video details the scope of counterfeit parts before regulation, the FAA’s own stock contained 39% counterfeit parts and other part brokers were much worse, and even air force one had counterfeit parts.
I expect (or at least hope) the aviation and medical industries have got a handle on this problem, but it wouldn’t surprise me at all if many consumer products on ebay and amazon are affected today and going completely under the radar.