It's not super clear to me what kind of competitive edge that is worth to Intel, as there are very few workloads that need hundreds of cores in a single OS image. There's advantages to reducing the number of ports etc you need in your datacenter network but I can't see that extending to preferring double the sockets and probably more than double the power.
cloudy skies: Intel's future in the datacenter
- Thread starter Megalodon
- Start date
It's not super clear to me what kind of competitive edge that is worth to Intel, as there are very few workloads that need hundreds of cores in a single OS image. There's advantages to reducing the number of ports etc you need in your datacenter network but I can't see that extending to preferring double the sockets and probably more than double the power.
Looking at the naming schemes it seems like more the latter, but yields may play a role as well. They have models designed for longevity / harsh environments that are more IoT- or appliance-like. There are mainstream 1S parts, there are parts for HPC, there are parts for multi-socket, there are parts optimized for network loads etc...
I'm sure they have a very robust marketing team figuring out which features are worth how much $$$ to which kinds of buyers-- and secondarily they may be aligning that with the chips that are actually able to be made.
For virtualized and cloud workloads, having lots of cores is still very helpful and since you're probably running virtualized hardware, and having hardware accelerators is much less useful. In that case the hypervisor may benefit from lots of cores, but the containers and guests running don't need to see all of them at once. The density consolidation is a big deal, especially if the heat and power requirements remain manageable.
It's those aggressively multi-socket systems that get ugly. You're not going to fit that on to a blade or a 1U-2U unit like you can with a 1S or 2S system. They're going to guzzle power and run ridiculously hot. I'm sure there are workloads where Intel thinks that some customer wants some monster server with 480 cores but most customers who need 480 cores total are more than happy to spread that load out across multiple systems. It's just that with Genoa, they can do that with ~5U of rack space or less which looks like a huge reduction over the ~12+ Units needed to do that many cores with Ice Lake (never mind the benefits of Genoa, just looking at raw core counts.)
Intel did say that for servers where bare metal is highly preferred (I can see that for networking-oriented hardware for example, and the obvious example of big database servers) they expect that the bulk of what they sell to be 32-core and below as that lines up with per-server-licensing models for big companies like Microsoft. That was one of the gotchas with Ice Lake. Licensing commercial server software can be really expensive and going from a moderately expensive 32-core-per-socket or 32-core-per-system license to an "unlimited" license would get really expensive, really quickly but you're only getting max 40 cores. Those top end parts languished. For Genoa you can justify that if you're paying more at least you're getting a lot more cores. Sapphire Rapids gets closer to that with SKUs that can exceed 32-cores by a meaningful margin but that's probably not where the bulk of sales by units shipped are likely to be.
One other point that I'm not sure the article covers since it's mostly about Sapphire Rapids: There's a lot of engineering going into all these different SKUs and whenever the next Intel platform arrives it's likely to be different still. The Genoa refresh parts and the Bergamo parts should all be drop-in replacements for Genoa. The board partners/OEMs will have to work with AMD to validate their boards and get updated firmware but that's much cheaper than validating an entire new platform.
The server market is way down right now, along with a lot of tech stuff and Intel really missed the timing on launching this. Had they hit in late 2021 or early 2022 as hoped, they would have launched into a much more robust market that was frankly consuming every server CPU that could be made with customers on 6-month waitlists or more in 2021. In that climate they wouldn't have even had to be all that great. AMD has made a lot of traction in the last 2 years and what parts they are producing have a lot of models that are well positioned to show their value (relative to Intel.)
True. There will also always be some wasted capacity because scheduling heterogeneous jobs across heterogenous hardware is hard (essentially the Knapsack Problem). The waste is reduced when individual systems have many more cores than needed for any given job. But that's very different than having a single job that won't work unless it has hundreds of cores as a single image, of which there are exceedingly few. It's an economic optimization problem, and when AMD's economics have such a huge advantage I think they win anyway.
There was a huge number of datacenter rebuilds and upgrades over the last decade as core counts increased for this reason, and Intel was by far the biggest beneficiary of this. Intel's stalled progress took a long time to give AMD enough of an edge.
Plus it's hard to overlook how conservative business customers are. AMD has a double-digit percentage of new server sales, but Intel still dwarfs them in both install base and new units sold. The main difference is that the average price per CPU on Intel is stagnating as customers don't want as many of their high end options and increasing on AMD as the big Epyc chips are still reasonably cost-effective (it's OK if a chip costs ~90% more if it has 100% more cores and you only need to buy a fraction as many server boxes as you would with Intel, etc.) For that reason, Intel is still important because while some business customers are highly focused on ROI other have other concerns and big OEMs don't want to run afoul of Intel or their customers by embracing AMD too exclusively.
AMD didn't help itself by having a whole lot of nothing for a large chunk of the last decade either, so acclimating businesses to buying Epyc based servers did take a while and is still ongoing.
AMD didn't help itself by having a whole lot of nothing for a large chunk of the last decade either, so acclimating businesses to buying Epyc based servers did take a while and is still ongoing.
I'd actually disagree on that. I think what we're seeing, namely very aggressive pricing from Intel on lower end SKUs, is suggestive of AMD having no difficulty at all selling out their entire TSMC capacity in high end SKUs. Merchant fab capacity is the bottleneck and that is slow enough to increase that it's not clear to me ramping up customers is a consideration at all. Smaller customers might be reluctant but if AMD wouldn't have the fab capacity to sell to them either way it's not really a barrier.
I think the reality is that throughout the entire Zen/Epyc era there's been a certain amount of risk-reduction adoption from very large customers like cloud providers etc. So they know the chips work and they're ready to ramp up whenever the economics support it.
The interesting thing with AMD is that a decade ago we were discussing how the company was a shambles with a terrible corporate structure and incapable of breaking out of a death spiral.
It's amazing that they were able to figure it out.
It's amazing that they were able to figure it out.
I guess what I'm trying to say is that AMD had some fairly promising servers around 2010-2012 if all you needed were lots of relatively slow cores-- the *"Magny-Cours" and later bulldozer/piledriver chips. They weren't great, weren't universally loved in the market, but they were serious attempts to try to compete and then... nothing. There were a few oddball processors here and there like AMD's A1100 ARM-based Opteron that was then immediately canned after its introduction in 2016. Until the first Epyc launched in 2017, AMD had basically nothing from about ~2012ish through 2017 and the 1st gen Naples Epyc CPUs were not the slam dunk that later generations are.
The mindshare/server market has 3 main constraints: Fab capacity, willingness of server customers to buy your processors and willingness of Server manufacturers (probably more conservative than customers) to build entire platforms and product lines around AMD processors. At first, customers demanded Epyc CPUs but the Dells and HPs and Supermicros of the world weren't super eager to jump into the market for Epyc because they remembered being burned when AMD promised a bright future in 2010 and then was dwindling in the market two years later-- ceding the market to Intel for half a decade.
Getting over all of these constraints is ongoing obviously but a lot of the statistics conflate install base with number sold in a given month/quarter/year. For the install base numbers the oldest Epyc CPUs are starting to drop off due to coming up for hardware refresh but if AMD had been able to hit the market harder in the beginning the install base numbers would probably be slightly higher and the sales rates might be higher (except for being fab constrained for at least part of that time.)
You can at least partially explain the increase in average selling price for AMD server chips since the instroduction of Epyc by talking about how AMD is fab constrained and is primarily offering higher profit, higher tier SKUs and that's part of it. Part of it is customers being willing to buy those SKUs at those prices. However that doesn't explain why Intel chips' ASP is getting lower. Intel's ASP going cheaper is only explainable by A) AMD's competition and AMD selling better products and B) Intel's products being weak enough they have to sell more bargain-oriented CPUs.
--
*This is the best pun in all of CPU naming, back when the homophone to "Many cores" topped out at 12 across 2 dies on a single CPU, when Intel's best was Nehalem based and topped out at 6 cores. It's hard to think that a little over a decade ago, 12 cores per CPU, two sockets per server-- a 24-core server was a beast. Now you can get that on the desktop.
I'm not sure this model works anymore. Most businesses aren't buying from Dell and HP these days, they're buying from AWS (possibly by way of a SaaS vendor), and AWS is its own OEM. So it matters a lot how the big cloud providers see things, and they're just fine with AMD.
Dell and HP etc might be trailing adopters but as long as that's not a bottleneck and AMD is saturating their TSMC capacity with high end SKUs, I don't think that's a problem for AMD.
I mean, the two things are corollaries, no? B is the logical result of A. There's no doubt some tweaking on Intel's end to try to maximize revenue given the constraints, but I don't think the specifics are material to the overall impression.
First off,
There are plenty of buyers for COTS servers. The TAM isn't as big as it might have been pre-AWS, but it is a solid business. HP(E), not HP and Dell are going to keep chugging along.
Second of all, Cloud computing is a fools errand for the manufacturers and I would be shocked if they are buying anybody's highest end SKUS. And even if they are, rest assured that Amazon, Microsoft and Google are going to take every penny of your margin that they can. They have no problem pitting AMD and Intel against each other while fostering an ARM solution at the same time.
Point being neither company will be posting big profits based on public cloud sales. Revenue yes, profit no. It's the same reason HP and Dell got out of trying to build the custom servers for those companies. They couldn't make money doing it so don't chase the market.
Not denying that, I just don't think it's necessary for AMD to focus there in this context.
That was always true and remains true regardless of AMD's participation; indeed, if you remove AMD from the analysis sticking with x86 looks appreciably worse from the cloud provider's perspective. So whatever the larger strategic concerns, AMD's decision is still simple. Moreover, there's nothing stopping AMD from adopting ARM (or RISC-V) should the market shift in that direction, and a lot of reasons to think an AMD-designed ARM core targeting high end servers would be competitive.
Depends a lot on the competitive landscape, I think. When you're as far ahead as AMD is right now I think there's still room for profitability. Much of Intel's problems there come from the fact that they were stuck on 14nm essentially forever, so they had to cut prices to the bone just to make it worthwhile for companies to refresh their own old Intel servers.
2022 Q4 was a no good, very bad quarter for Intel.
The lowlights:
I won't bother going through their numbers myself, but links to news analysis:
www.cnbc.com
www.cnbc.com
www.cnbc.com
Also, Intel is exiting the network switch business, and they have ended their RISC-V Pathfinder program.
The lowlights:
- Revenue is way down YoY, particularly in client.
- Gross margin is way down YoY across basically the entire business.
- The AXP group continues to burn money for Intel. While it's not losing as much money as in past quarters, the launch of Arc didn't move the needle much.
- One of the things weighing on gross margins is that Intel's fabs are spending time idling (i.e., manufacturing capacity is exceeding demand). Given that Intel has invested billions in expanding its fabs, and it being a key plank of Gelsinger's turnaround strategy, this doesn't bode well.
- Intel's guidance for Q1 2023 was pessimistic (as bad or worse than 2022 Q4), and they didn't even bother providing outlook for the rest of the year.
I won't bother going through their numbers myself, but links to news analysis:
Intel's horrible quarter revealed an inventory glut and underused factories
"Clearly, the financials aren't what we would hoped," Gelsinger told analysts.
www.cnbc.com
Intel shares drop on weak quarterly results and expectations for another quarter of losses
Intel called for a loss in the first quarter, while analysts had expected income. The chipmaker is dealing with a sharper-than-expected PC market contraction.
www.cnbc.com
Intel stock tumbles after brutal results
Intel shares fell Friday, after the company reported horrid quarterly and full-year results, off slackening chip demand.
www.cnbc.com
Also, Intel is exiting the network switch business, and they have ended their RISC-V Pathfinder program.
Just saw an alert on my phone that intel is slashing salaries for people over midlevel by at least 5%. Also some perks dropped
Yup. More here
www.semianalysis.com
It's like they want their best employees to leave
Intel Cuts Pay For Employees To Keep Their Quarterly Dividend
Hot off the presses, we have confirmation from multiple employees that Intel is cutting costs tremendously at the expense of their employees. These cuts are broad-based, with employees having their compensation affected. Quarterly pay bonuses are gone, annual bonuses are being paused, 401k match...
www.semianalysis.com
It's like they want their best employees to leave
Well, I mean Intel always had a "Unique" culture, by which I mean a culture that normal people would never tolerate.
So I figure if you're already above mid level at the company, this kind of torture probably feels like an upgrade.
So I figure if you're already above mid level at the company, this kind of torture probably feels like an upgrade.
Meanwhile, at AMD, they exceeded expectations for Q4. Client is down due to overall weakness across the market, but AMD is cleaning up in the data center right now (although DC spend as a whole seems to be shrinking). The Xilinx acquisition also looks like it's going well, and is contributing positively to AMD's numbers.
I guess we'll see if Sapphire Rapids makes a difference in H1 2023, but things continue to look dire for Intel's DC prospects, now and for the foreseeable future.
I guess we'll see if Sapphire Rapids makes a difference in H1 2023, but things continue to look dire for Intel's DC prospects, now and for the foreseeable future.
Sapphire Rapids is on the Intel 7 node right? Whereas AMD has been shipping TSMC N5 for a while. And then Intel is targeting 2024 for the Intel 4 node, but AMD will be launching N3 around then. So this does seem like the gap is modestly narrowing, depending on the exact details of Intel 4 vs TSMC's N3 and other implementation details. Sounds like that would mean Intel being more or less a half-node behind, rather than a full node or more.
AMD Grabs Over 30% CPU Market Share As Intel Continues To Decline
Also, AMD's server CPU market share specifically is expected to exceed 30% by the end of 2023.
Also, AMD's server CPU market share specifically is expected to exceed 30% by the end of 2023.
Looks like Intel is finally cutting its dividend. It's not directly related to the data center topic in thread, but is further evidence that Intel is under financial duress, and is taking steps to stop the bleeding.
Intel slashes dividend by over 65%, to 12.5 cents
Intel slashes dividend by over 65%, to 12.5 cents
CNBC said:
Microsoft is already providing Epyc as IAAS: https://azure.microsoft.com/en-us/partners/directory/amd-corporation/. However what will be more interesting in the future is them replacing their PAAS server blades that run using Intel Xeon with AMD Epyc chips.
More cancelled products from Intel: RIP Rialto Bridge
So much for Gelsinger being able to reduce the roadmap churn.
So much for Gelsinger being able to reduce the roadmap churn.
Intel is exiting their server business
(To be clear, this refers to Intel-branded fully-assembled servers, a la Supermicro, Dell, HPE, etc.)
(To be clear, this refers to Intel-branded fully-assembled servers, a la Supermicro, Dell, HPE, etc.)
ServeTheHome said:
I guess this means AMD Epyc is going to dominate server farms very soon, as Intel is exiting the business.
Intel is exiting the finished serve business. not the server processor business.
Intel loses $2.8 billion in another not-great quarter, down 36% year over year
And Intel's leadership says it will still take time to turn things around.
arstechnica.com
Client revenue down 38%, server revenue down 39%. Bloodbath.
Not just that. It's their 5th straight quarter of decline, their lowest year since 2018 and their lowest quarter since 2010.Intel loses $2.8 billion in another not-great quarter, down 36% year over year
And Intel's leadership says it will still take time to turn things around.
Client revenue down 38%, server revenue down 39%. Bloodbath.
Meanwhile AMD just posted their highest revenue ever, and ARM architecture continues to make inroads into markets dominated by x86.
I'd wait until after AMD reports next Tuesday to make that comparison. All signs point to this being an industry-wide slump.
They've looked healthier than Intel in 2022.
edit:
my previous post was looking at trailing 4 quarters, which can conceal recent trends. their profits were record for the trailing 4 quarters, but off compared to 1q of last year.
Looks like datacenter was flat (vs down sharply for Intel), gaming down modestly, and embedded up sharply (??? that's new?). Overall revenue down but looking a hell of a lot better than Intel.
AMD's client market has completely cratered, and while growth was flat in data center, its operating income was way down in that segment. AMD claimed that this was caused by weakness in the enterprise sector, which caused them to rely more on sales from cloud buyers where margins are much lower.
When taking questions from analysts, many of them were related to where AMD was at on AI, and it was clear they didn't have a great story there. Nvidia obviously owns AI workloads running on GPUs, and Intel has acceleration for AI running on CPUs, which leaves AMD is in a weak spot in the one area of the market that is still growing.
So overall, the story of Q1 is essentially a continuation of what we saw previously: AMD eating Intel's lunch in traditional DC workloads, while Intel makes gains elsewhere due to aggressive pricing.
However, the market has changed, and not in AMD's favor. If AMD wants to continue their success in the DC space, then they have got to get their shit together in the GPU market, and fast.
Not sure where Intel is going, either. They've got a few things brewing, but they're focusing on building fab capacity at a time when the semiconductor market has contracted significantly, and they're fighting a price war. I don't know how sustainable that will be.
Well, that was fast.
AMD jumps on report Microsoft is collaborating on A.I. chip push
Unlike top U.S. cloud competitors Amazon and Google, Microsoft hasn't launched a chip that developers can use to train artificial-intelligence models.
www.cnbc.com
More layoffs at Intel, with the data center group supposedly being among the impacted.
www.tomshardware.com
Intel Comments on New Layoffs, Budget Cuts in Client CPU and Data Center Groups
Intel is reducing its workforce.
Tom's Hardware said:
While Sapphire Rapids can't match the raw general-purpose computational throughput of Genoa, Intel's selling point has been the additional acceleration functionality that Sapphire Rapids has to offer, in the form of on-board hardware accelerators, extensions like AMX, on-board HBM, etc.
Phoronix put an HBM2e-equipped Xeon Max 9480 through some OpenVINO benchmarks, which is a workload that can take advantage of AMX and HBM, and the results are quite impressive. The results with HBM are in a league of their own, and even without HBM, the Xeon Max 9480 outperforms the EPYCs while consuming less power.
It remains to be seen how the performance picture will change as more software starts taking advantage of Sapphire Rapids' functionality, or how much it will matter in practice vs. just running these types of workloads on a GPU. However, a win is a win, and Intel's chips have put on a good showing here.
Phoronix put an HBM2e-equipped Xeon Max 9480 through some OpenVINO benchmarks, which is a workload that can take advantage of AMX and HBM, and the results are quite impressive. The results with HBM are in a league of their own, and even without HBM, the Xeon Max 9480 outperforms the EPYCs while consuming less power.
It remains to be seen how the performance picture will change as more software starts taking advantage of Sapphire Rapids' functionality, or how much it will matter in practice vs. just running these types of workloads on a GPU. However, a win is a win, and Intel's chips have put on a good showing here.
Curious how the extra cache in Genoa-X impacts this as that adds the stacked cache and is known to be quite beneficial for HPC workloads. Genoa-X SKUs do not appear to be part of the benchmarks presented in the link. That seems like a strange omission for this workload.
Intel does seem to be returning to a better posture overall though. Sapphire Rapids should be spending a lot less power on the fabric connections than AMD.
While this isn't directly related to Intel's DC products, Intel is winding down its NUC business: https://www.servethehome.com/intel-exiting-the-pc-business-as-it-stops-investment-in-the-intel-nuc/
Direct quote from Intel:
Direct quote from Intel:
Intel said:
I saw that and my first thought was it makes a lot of sense. I don't consider NUC very often anymore as there are other vendors out there offering the same product often cheaper. I really like the MSI CUBI products
I have to look at this positively even though I hate to see Intel vacate a market here that they literally created from scratch (the NUC.)
They're clearly over-extended and most of the recent divestitures are in areas where they have partners who can immediately move in and fill the gap. Intel will still sell Xeons, just not their own white box servers. Intel will still sell various low-power CPUs that go into mini computers, they just won't sell NUCs. The abandonment of the HEDT range of processors on desktop allows them to hopefully simplify and offer lower-end Xeon parts in workstations -- but the range of people who wanted the greater core count of HEDT and the greater bandwidth was shrinking and getting pinched between desktop parts packing in more cores and bigger workloads being offloaded to servers or even cloud-based resources. The unfortunate reality in that space is that Intel might only capture part of that vacated market with Sapphire Rapids Xeons. Threadripper (Pro) is going to probably get some percent of that market. While not a divestiture, the disconnection of the fab from the design side of the house is also a strategic move to make their internal groups think more about efficiency and try to ironically get them to collaborate more before hand if the design folks will be charged back now for limited silicon test runs.
If that frees up Intel to be more nimble-- to continue to refine process nodes and eventually get more performance/watt capable chips that are not just glorified mobile parts -- to maybe push the accelerator concept and iterate on the highly uneven (and complicated) Sapphire rapids lineup and allows them to keep ARC alive as a strategic investment -- useful now for consumer but down the road having that AI-capable tech on a GPU is going to position them against nVidia in the datacenter*-- then I'm OK with divesting the NUC lineup, cancelling HEDT, etc.
it's not clear how well Grace Hopper and the various integrations of nVidia CPU *and GPU tech in the same platform will pay out, but I know the big server OEMs are nervous and highly annoyed with nVidia as a partner. For now there are plenty of GPU-compute/AI-compute systems in the market that have nVidia hardware running on a Xeon or Epyc, but nVidia is sending signals that they want to cut out the one part of the platform they don't currently control. AMD's not fully competitive but they offer something. Intel's playing catch up during a time when their financial resources are highly constrained but if they don't get that GPGPU tech refined soon, there a definite risk of being relegated to only certain niches in the datacenter.
They're clearly over-extended and most of the recent divestitures are in areas where they have partners who can immediately move in and fill the gap. Intel will still sell Xeons, just not their own white box servers. Intel will still sell various low-power CPUs that go into mini computers, they just won't sell NUCs. The abandonment of the HEDT range of processors on desktop allows them to hopefully simplify and offer lower-end Xeon parts in workstations -- but the range of people who wanted the greater core count of HEDT and the greater bandwidth was shrinking and getting pinched between desktop parts packing in more cores and bigger workloads being offloaded to servers or even cloud-based resources. The unfortunate reality in that space is that Intel might only capture part of that vacated market with Sapphire Rapids Xeons. Threadripper (Pro) is going to probably get some percent of that market. While not a divestiture, the disconnection of the fab from the design side of the house is also a strategic move to make their internal groups think more about efficiency and try to ironically get them to collaborate more before hand if the design folks will be charged back now for limited silicon test runs.
If that frees up Intel to be more nimble-- to continue to refine process nodes and eventually get more performance/watt capable chips that are not just glorified mobile parts -- to maybe push the accelerator concept and iterate on the highly uneven (and complicated) Sapphire rapids lineup and allows them to keep ARC alive as a strategic investment -- useful now for consumer but down the road having that AI-capable tech on a GPU is going to position them against nVidia in the datacenter*-- then I'm OK with divesting the NUC lineup, cancelling HEDT, etc.
it's not clear how well Grace Hopper and the various integrations of nVidia CPU *and GPU tech in the same platform will pay out, but I know the big server OEMs are nervous and highly annoyed with nVidia as a partner. For now there are plenty of GPU-compute/AI-compute systems in the market that have nVidia hardware running on a Xeon or Epyc, but nVidia is sending signals that they want to cut out the one part of the platform they don't currently control. AMD's not fully competitive but they offer something. Intel's playing catch up during a time when their financial resources are highly constrained but if they don't get that GPGPU tech refined soon, there a definite risk of being relegated to only certain niches in the datacenter.
Missing out on AI is hurting Intel a lot.
nvidianews.nvidia.com
NVIDIA Announces Financial Results for Second Quarter Fiscal 2024
NVIDIA today reported revenue for the second quarter ended July 30, 2023, of $13.51 billion, up 101% from a year ago and up 88% from the previous quarter.
MS just announced two in-house chips - one AI and one CPU - to power Azure. Yet one more contract that Intel isn’t going to win. It will be interesting to see how these sorts of developments erode Intel’s server profitability.
That said Microsoft's capital expenditure plans are so huge that even with their own chips they will still be spending a fortune with Intel, AMD, and Nvidia; and I dare say that even Ampere is still likely to sell a decent amount to Microsoft.