WWDC, June 10-14, 2024, to be 'A(bsolutely) I(ncredible)' — Keynote Discussion Here!

Jump to latest Follow Reply
japtor

japtor

Ars Legatus Legionis
13,043
  • #321
    jaberg said:
    One more nail in 1PWs coffin—Only family sharing remains.
    Click to expand...
    Didn't they add sharing last year or so?
    Hap said:
    They could already, but only if the webpage coded for it. I have a couple of sites it just works on, but only a couple. Hell, some sites STILL block password paste.
    Click to expand...
    I haven't run into that too much...but have one particular instance that's really annoying, I think it's Bank of America's login. Tapping the auto fill just makes it not work for whatever reason, like I have to get the OTP again and just manually type it or copy/paste from Messages (can't just copy from the shortcut argh). I'd guess it has something to do with the auto submit part of the equation...which is something that's been annoying here and there as well in general. While convenient, I really wish it didn't do that, hoping one of these days it'll be a silently added setting.
    ev9_tarantula said:
    Something that occurs to me is that if Apple wants Apple Intelligence to be available on all new iPhones the next SE will be a huge upgrade, as it would be really odd to launch a new phone that can't run the main new software features. Maybe not using the A17 Pro, but perhaps an A18 with the newer NPU architecture, and 8 GB to allow local models to run. Mind you even 8 GB is pushing it for current models, that's going to potentially cause a lot of problems with devices in future.
    Click to expand...
    As someone with an iPhone 13 mini and iPad mini I was thinking the same thing, my next upgrades will hopefully be nice...whenever the hell that happens. And yes there will be a new mini iPhone one day!*

    *by the time they make a new one, it'll be the size of the Pro Max but still be the smallest of the lineup when everything else is approaching being iPad micros.
    jozero said:
    That was AI super well integrated. Knowledge Navigator is finally here, they should of gave Siri icon a bow tie!!
    Click to expand...
    The new icon is kind of bow tie like 🤔
     
    Last edited:
    W

    wco81

    Ars Legatus Legionis
    28,661
  • #322
    jaberg said:
    For me, this feature might not be the biggest of deals, but it is welcome never the less.

    On any given day there’s a better than even chance that the iPhone will sit on its charger next to the bed until I leave the house. At the very least, it will be there until I get dressed…because pockets are requisite for carrying the phone around. (I have need of my hands for other tasks.) Care for my senior father has broken of me of my life-long morning routine which included dressing first thing in the morning.

    In terms of the trek — it would have been more of one in my previous home, a (small) Victorian mansion where I was frequently working in an adjacent carriage house. Here it’s not the distance but rather the likelihood of being side-tracked by The Elders anytime I leave the sanctity of my office/studio area. Or that I’ve forgotton the phone in the car, and will ultimately be calling upon Find My to locate it. Both risk my losing the flow of whatever task I needed the phone for to begin with.

    Not having the phone on my person is made possibile because I’ve been able to answer the phone from Mac or iPad for some time now. Having full access to the phone from the desk, in the basement, is absolutely something that will be useful and helpful to me.

    What I don’t get is why we contestantly need to reiterate NOT FOR YOU ≠ NOT FOR ANYONE.
    Click to expand...

    Oh I'm glad for the feature, just surprised at how enthusiastic people are for it.

    Happy that many people will find it useful, if not indispensable.
     
    W

    wco81

    Ars Legatus Legionis
    28,661
  • #323
    Gandhim3 said:
    Market certainly likes what they saw after they had time to digest the information Overnight
    Click to expand...
    Yeah that's some 180 from yesterday.

    Couple of stock market pundits saying yes this will drive a big upgrade cycle.

    I think there are some high number of iPhones which are 3-4 years old or older out there.

    We will see if people upgrade specifically for Apple Intelligence or cite it as one of the main reasons. One thing that works in Apple's favor is that the AI hype is in overdrive right now.

    Even people who casually follow technology will have heard and seen multiple mentions of AI.
     
    jaberg

    jaberg

    Ars Praefectus
    3,660
    Subscriptor
  • #326
    japtor said:
    Didn't they add sharing last year or so?
    Click to expand...
    They probably did — I can't remember exactly why I didn't switch at that time, but if memory serves it wasn't good enough to push me off of 1PW.

    Thinking it through, the issue may have been that it was only this year that I added The Elders to a shared family iCloud account. I should revisit the option and perhaps reconsider.
     
    G

    grahamb

    Ars Praefectus
    3,623
  • #327
    Last edited:
    DovePig

    DovePig

    Ars Praetorian
    10,283
  • #328
    • Like
    Reactions: grahamb
    iPilot05

    iPilot05

    Ars Tribunus Militum
    2,780
    Subscriptor
  • #329
    wco81 said:
    I think there are some high number of iPhones which are 3-4 years old or older out there.

    We will see if people upgrade specifically for Apple Intelligence or cite it as one of the main reasons. One thing that works in Apple's favor is that the AI hype is in overdrive right now.

    Even people who casually follow technology will have heard and seen multiple mentions of AI.
    Click to expand...
    The massive supercycle from covid was about that long ago so I imagine, yes, there's a lot of people with 2020-2021 era phones out there. Especially casual users like my parents with iPhone 11s just purring away like they are brand new. If Apple can really market Apple Intelligence as the "AI for the rest of us" it might really spur some folks to upgrade even those with 1-2 year old lower-spec phones.

    I bet it'll also be a final nail in the coffin for those on Intel Macs. At this point even the ones that support Sequoia will have a noticeably hobbled experience running it compared to even a base-spec M1 Mac.
     
    S

    Scotttheking

    Ars Legatus Legionis
    10,928
    Subscriptor++
  • #333
    stevenkan said:
    Can you share how you generated that estimate?

    How many users can you serve with a node? How do nodes map to physical servers?

    So many questions . . . .
    Click to expand...
    Sure. I'm making this all up though :). Here's an attempt to take my WAG and put some numbers.

    Node - a unit with compute - I'm guessing it's a blade with a CPU, memory, storage, network. Think server.

    Assumptions (the first 3 of which I'm completely making up):
    a node has 8 cores/can handle 8 simultaneous requests
    a request takes around 5 seconds on average
    an average user sends two requests per hour on average
    there are 500 million users

    8 simultaneous requests at 5 seconds per request = 1.6 requests per second per node
    500,000,000 * 2 = 1,000,000,000 requests per hour
    divide by 3600 seconds per hour yields = 277,777 requests per second
    divide by 1.6 requests per node = 173,361 nodes needed

    Add some scaling factor, redundancy: 200,000 nodes


    That ignores model training and such. And I made this all up. But that's running numbers behind the initial guess.
     
    J

    johnsonwax

    Ars Legatus Legionis
    14,407
  • #334
    Scotttheking said:
    Sure. I'm making this all up though :). Here's an attempt to take my WAG and put some numbers.

    Node - a unit with compute - I'm guessing it's a blade with a CPU, memory, storage, network. Think server.

    Assumptions (the first 3 of which I'm completely making up):
    a node has 8 cores/can handle 8 simultaneous requests
    a request takes around 5 seconds on average
    an average user sends two requests per hour on average
    there are 500 million users

    8 simultaneous requests at 5 seconds per request = 1.6 requests per second per node
    500,000,000 * 2 = 1,000,000,000 requests per hour
    divide by 3600 seconds per hour yields = 277,777 requests per second
    divide by 1.6 requests per node = 173,361 nodes needed

    Add some scaling factor, redundancy: 200,000 nodes


    That ignores model training and such. And I made this all up. But that's running numbers behind the initial guess.
    Click to expand...
    It'll be way less than 2 requests per hour. My guess is that most users will never trigger a server request - everything they want to do can be done on-device. It'll be stuff like larger image generation that triggers it. The requests will take longer given how the servers work - they reboot the instance between requests to clear out memory, get back to a pristine state. Guessing you're at least an order of magnitude too high.
     
    stevenkan

    stevenkan

    Ars Legatus Legionis
    15,662
  • #335
    johnsonwax said:
    It'll be way less than 2 requests per hour. My guess is that most users will never trigger a server request - everything they want to do can be done on-device. It'll be stuff like larger image generation that triggers it. The requests will take longer given how the servers work - they reboot the instance between requests to clear out memory, get back to a pristine state. Guessing you're at least an order of magnitude too high.
    Click to expand...
    But on the flip side, I can't imagine Apple designing custom hardware, and possibly custom silicon, for 20,000 units of anything.

    edit: so if it's <<< 100,000 units, I think they'd probably run this on stock Mac Pro or Studio units, while they gauge ultimate demand/load.
     
    • Like
    Reactions: The Limey
    E

    ev9_tarantula

    Ars Praefectus
    5,221
  • #336
    stevenkan said:
    But on the flip side, I can't imagine Apple designing custom hardware, and possibly custom silicon, for 20,000 units of anything.
    Click to expand...

    Doing some back of the envelope calculations I think Google might have somewhere in the region of 1.6 million liquid-cooled TPUs, so 400,000 nodes. On top of that they have air-cooled TPUs for smaller deployments at a wider range of data centres. I'd guess that would double their fleet or more. So it's quite plausible that Google has 4 million TPUs in total, and 1 million nodes*.

    Now I doubt Apple's setup would be as large, but probably at least 10% of that size, and maybe up to half it if they went really all-in on AI.

    * If this sounds ridiculous note that Meta are said to have spent perhaps $30 billion on 1 million Nvidia GPUs, and Microsoft and OpenAI are planning a supercomputer(s) called Stargate that might cost as much as $100 billion and use 5 GW of power.
     
    G

    gregatron5

    Ars Legatus Legionis
    11,245
    Subscriptor++
  • #337
    nimro said:
    From Apple's page on it's model training:


    So, they never use their users' private personal data... unless their web crawler hoovers it up. Same scummy opt-out data vacuuming that every other shitty corporate AI play is relying on.

    There goes all hope of I had Apple doing it "better". It's the same trend-chasing consent-disregarding crap everyone else is doing they're just late to the party.
    Click to expand...
    This is an extremely ungenerous take. Apple's opt-out is robots.txt, which has been the standard way to opt out of indexing since… forever (in internet time).
     
    • Like
    Reactions: Galberras, eas, gabemaroz and 3 others
    G

    greatlakeexpat

    Ars Centurion
    357
  • #338
    ev9_tarantula said:
    What happens when the model grows?

    Apple has simply been too cheap with RAM on their devices for many years, and now they are running head first into a use case where capacity and bandwidth really matter.
    Click to expand...
    Make people upgrade more often or raise prices to make up the difference if the AI bits take off.

    I’m likely not going to upgrade from my 13 pro, but I will run Apple’s AI on my iPad or MBP in the fall, but I would rue the day if it makes my new iPad feel sluggish if the on device LLM chew up half the device’s memory.

    If it does chew up memory, please let me disable it as a workaround if I’m not getting a good experience or using it much.

    If AI does become beneficial, then sure, an upgrade cycle may happen sooner, but it could also lead to longer ones if the technology keeps rapidly leapfrogging.
     
    C

    Chris FOM

    Ars Legatus Legionis
    10,001
    Subscriptor
  • #339
    stevenkan said:
    But on the flip side, I can't imagine Apple designing custom hardware, and possibly custom silicon, for 20,000 units of anything.

    edit: so if it's <<< 100,000 units, I think they'd probably run this on stock Mac Pro or Studio units, while they gauge ultimate demand/load.
    Click to expand...
    R1? Volume on that can’t be very high at all. And if their AI stuff is an add-on rather than part of the SoC there’s at least the possibility that Apple could stick it on a PCI card and sell it for the Mac Pro.
     
    • Like
    Reactions: Bonusround and The Limey
    B

    Bonusround

    Ars Scholae Palatinae
    1,060
    Subscriptor
  • #340
    Chris FOM said:
    R1? Volume on that can’t be very high at all. And if their AI stuff is an add-on rather than part of the SoC there’s at least the possibility that Apple could stick it on a PCI card and sell it for the Mac Pro.
    Click to expand...
    The R1 and all DTKs throughout the years. 20K units is sufficient to escape the harsh economics of one-off or prototype units. And even at five-figure volumes I suspect Apple know one or two companies that might be motivated to cut them a deal.

    And don’t forget, any such machine will be populated with components (the vast majority of which) Apple already acquire in significant quantities. And I’ll venture to guess that, given this hardware is squirreled away out of public sight, Apple have no qualms using standard, inexpensive rack mount casing. No exotic CNC Mac Pro rack required here.
     
    • Like
    Reactions: The Limey
    C

    Chris FOM

    Ars Legatus Legionis
    10,001
    Subscriptor
  • #341
    Bonusround said:
    The R1 and all DTKs throughout the years. Even 20K units is sufficient to escape the harsh economics of one-off or prototype units. Even at five-figure volumes, I suspect Apple knows one or two companies that might be motivated to cut them a deal.

    And don’t forget, any such machine will be populated with components (the vast majority of which) Apple already acquire in significant quantities. And I’ll venture to guess that, given these squirreled away out of public sight, Apple have no qualms using stock, inexpensive rack mount casing. No exotic CNC Mac Pro rack required here.
    Click to expand...
    All true, but in this case the specific discussion is for the silicon itself. By far the biggest fixed cost for chips is the mask, and those costs are big enough (nine figures if I remember what I’ve read correctly) even a company of Apple’s size will notice. And the only way around that is either clever reuse (like the M1/2 Pro/Max/Ultra getting three different variants out of one mask) or using an older node. It was heavily commented on at the time that the M3 Pro and Max were completely different dies instead of the Pro being a chopped Max like the previous gen. Don’t forget the Apple Silicon DTKs used an A12Z, not a custom chip. They were basically an iPad Pro in a Mac mini enclosure.
     
    B

    Bonusround

    Ars Scholae Palatinae
    1,060
    Subscriptor
  • #342
    Chris FOM said:
    All true, but in this case the specific discussion is for the silicon itself. By far the biggest fixed cost for chips is the mask, and those costs are big enough (nine figures if I remember what I’ve read correctly) even a company of Apple’s size will notice. And the only way around that is either clever reuse (like the M1/2 Pro/Max/Ultra getting three different variants out of one mask) or using an older node. It was heavily commented on at the time that the M3 Pro and Max were completely different dies instead of the Pro being a chopped Max like the previous gen. Don’t forget the Apple Silicon DTKs used an A12Z, not a custom chip. They were basically an iPad Pro in a Mac mini enclosure.
    Click to expand...
    Thanks.

    With the power of hindsight in the wake of M4, we can now understand the missing UltraFusion interconnect on the M3 Max is most likely because Apple never planned to make M3 Ultras at all. The supposition that the Max’s missing fabric indicates a unique die for Ultra is now only that – supposition.

    On the subject of custom silicon costs, allow me to refer to and quote a prior post. Modern chip design is terribly expensive, but the masks themselves do not appear to be the dominant portion of that cost.

    Bonusround said:
    Incidentally, I came across an old Gartner report breaking down SoC production costs for 7nm, which at one time was planned to be the first node produced using EUV. (In reality TSMC only used EUV for a few layers on one of the 7nm variants – the vast majority were multi-patterned DUV lithography.) This report placed "Mask Cost" at 2.5% of the total, "Design Cost" was ~78%, "Embedded Software" ~12%, and "Yield Ramp-Up Cost" ~7.5%.

    The total dollar cost for creation of this notional 7nm SoC was just north of $600M in 2018 dollars. By contrast a 10nm SoC was cited as $400M in 2016, and a 16nm SoC at just below $300M in 2014 (where masks represented only 1.5% of the total). These chips are very expensive efforts, no question, and will surely have grown more expensive since, especially for full EUV nodes like 5nm and 3nm.
    Click to expand...
     
    W

    wrylachlan

    Ars Legatus Legionis
    12,769
    Subscriptor
  • #343
    Bonusround said:
    Thanks.

    With the power of hindsight in the wake of M4, we can now understand the missing UltraFusion interconnect on the M3 Max is most likely because Apple never planned to make M3 Ultras at all. The supposition that the Max’s missing fabric indicates a unique die for Ultra is now only that – supposition.

    On the subject of custom silicon costs, allow me to refer to and quote a prior post. Modern chip design is terribly expensive, but the masks themselves do not appear to be the dominant portion of that cost.
    Click to expand...
    I can’t speak for others but generally when I refer to mask cost I mean ‘the cost of getting to the point at which you have a completed mask’ not the cost of the mask as a physical object. That includes all of those costs that Gartner is identifying as design.

    That said, since the bulk of the cost of getting to the point of a functioning mask is design, it’s worth pointing out that not all designs are equally difficult. Designing a brand new core? Lots of time and effort. Laying out regular repeating structures? Not so much.

    The NPU likely falls into the latter category. A new mask that uses an existing couple P core clusters and then just tiles a shed load of NPU cores and some memory controllers is certainly on the lighter side of Apple’s design portfolio.

    It’s also worth noting that automatic layout tools have (like all the rest of the tech world) progressed substantially since 7nm.
     
    B

    Bonusround

    Ars Scholae Palatinae
    1,060
    Subscriptor
  • #344
    wrylachlan said:
    I can’t speak for others but generally when I refer to mask cost I mean ‘the cost of getting to the point at which you have a completed mask’ not the cost of the mask as a physical object. That includes all of those costs that Gartner is identifying as design.
    Click to expand...

    In the past I recall our discussing chip variants which would logically constitute a minimal design effort, but definitely demand a new mask. Into this scenario the specter of "mask costs!" was injected.

    Water under the bridge. Henceforth let's recognize the cost distinction between design effort and cutting the mask which captures that effort.

    And, as you note, this distinction will be especially relevant given the extensive adoption of automated layout (and other processes) by TSMC.

    wrylachlan said:
    The NPU likely falls into the latter category. A new mask that uses an existing couple P core clusters and then just tiles a shed load of NPU cores and some memory controllers is certainly on the lighter side of Apple’s design portfolio.
    Click to expand...

    No surprise that I'm all about this. :)
     
    J

    johnsonwax

    Ars Legatus Legionis
    14,407
  • #345
    stevenkan said:
    But on the flip side, I can't imagine Apple designing custom hardware, and possibly custom silicon, for 20,000 units of anything.

    edit: so if it's <<< 100,000 units, I think they'd probably run this on stock Mac Pro or Studio units, while they gauge ultimate demand/load.
    Click to expand...
    Apple is almost certainly counting on their ability to jam more compute on future processors to reduce the need for server-side compute. They did the same thing with Siri shifting from being entirely cloud-compute to now almost entirely device-compute. So it's a capital cost that they incur to roll the service out which has rapidly declining marginal costs.

    Even if their volume was high enough to justify custom silicon, my guess is their plan is that this is largely temporary.

    I'm guessing it's not stock units but a custom motherboard for data center. That's pretty cheap/fast to redesign around their existing components.
     
    J

    johnsonwax

    Ars Legatus Legionis
    14,407
  • #346
    Chris FOM said:
    R1? Volume on that can’t be very high at all. And if their AI stuff is an add-on rather than part of the SoC there’s at least the possibility that Apple could stick it on a PCI card and sell it for the Mac Pro.
    Click to expand...
    R1 doesn't need the volume because it doesn't have component economics. Apple can use as much of the margins on the whole $3500 AVP to pay for the R1 that allows the product to even exist. This is clearly a product segment Apple expects to lift into large enough volume to make R1 economics work out and they'll take any necessarily losses to get there.
     
    stevenkan

    stevenkan

    Ars Legatus Legionis
    15,662
  • #347
    johnsonwax said:
    Apple is almost certainly counting on their ability to jam more compute on future processors to reduce the need for server-side compute. They did the same thing with Siri shifting from being entirely cloud-compute to now almost entirely device-compute.
    Click to expand...
    Except that my iPhone still needs to talk to a datacenter to process "Yes" vs. "No." 🙄
     
    nimro

    nimro

    Ars Tribunus Militum
    2,097
    Subscriptor++
  • #348
    gregatron5 said:
    This is an extremely ungenerous take. Apple's opt-out is robots.txt, which has been the standard way to opt out of indexing since… forever (in internet time).
    Click to expand...
    Until this WWDC it was widely understood that AppleBot (Apple's crawler) was performing search indexing activities, not scraping data for model training. Those are two very different things and website owners might have appreciated some warning rather than being told the data was already in the model and they should have opted out of something months ago they had no knowledge of.

    Furthermore, please explain how I can use robots.txt to protect my personal data from Apple. I don't control most websites on the internet. Do I have to go around every website I have posted to in the last 25 years (and every website someone who knows me may have posted something on) and ask them to go back in time and alter their robots.txt a few months ago? How about sites hosting leaks and hacked dumps of sensitive data? Think they'll be willing to accomodate my preferences? Come on.

    Here's why I have a problem with it: Apple went to great pains to explain the extraordinary measures they are taking to protect personal data on customer devices. But if that same customer's data happens to be on the web for some reason? Well then they see it as fair game.

    It's shitty enough when OpenAI, Google, etc. are doing it and being open about it, but Apple is trying to present this image of shielding your data, while in the background hoovering up as much of that data as it can. Hypocrites.
     
    • Like
    Reactions: Honeybog, gabemaroz and phlaym
    A

    Aleamapper

    Ars Scholae Palatinae
    1,284
    Subscriptor
  • #349
    nimro said:
    Here's why I have a problem with it: Apple went to great pains to explain the extraordinary measures they are taking to protect personal data on customer devices. But if that same customer's data happens to be on the web for some reason? Well then they see it as fair game.
    Click to expand...
    What exactly do you mean by personal data? If you're talking about personally identifiable information, the Apple article @gregatron5 linked to quite clearly explains that PII is filtered from scraped content before being used for training.

    If you're talking about, eg a story you wrote that you posted on a public forum, then yeah, you're out of luck.
     
    • Like
    Reactions: Tagbert
    nimro

    nimro

    Ars Tribunus Militum
    2,097
    Subscriptor++
  • #350
    Aleamapper said:
    If you're talking about personally identifiable information, the Apple article @gregatron5 linked to quite clearly explains that PII is filtered from scraped content before being used for training.
    Click to expand...
    Yes PII that fits their automated processing expectations is filtered. If you've ever worked with any amount of raw data you will know that there is zero chance it's all been removed. Did they cover every worldwide format of every possible type of PII? Seems exceedingly unlikely.
     
    J

    johnsonwax

    Ars Legatus Legionis
    14,407
  • #351
    Aleamapper said:
    What exactly do you mean by personal data? If you're talking about personally identifiable information, the Apple article @gregatron5 linked to quite clearly explains that PII is filtered from scraped content before being used for training.

    If you're talking about, eg a story you wrote that you posted on a public forum, then yeah, you're out of luck.
    Click to expand...
    Yeah, the issue is simply this:
    Ars Technica reserves the right to reproduce, edit, remove, or distribute any post contained in the OpenForum. All posts become the property of Ars Technica, except when copyrighted material is posted (and properly cited).
    Click to expand...
    As such, Ars Technica has the final say on whether our comments are scraped or licensed - and I don't know of place you can post that doesn't have the same T&C. Now, if you have a xmpRights:UsageTerms with a non-commercial CC license, for instance, set in a PNG you post, then presumably AppleBot should be able to exclude that, but who the hell knows, and if they don't exclude it, there's not much real opportunity for you to cease and desist them from training off of it. It would be nice if there was considerably more transparency in that process.

    Hacked data is already illicit, so Apple is breaking the law by even using it. Probably safe to say they are smart enough not to. Won't predict any other parties on that one though.
     
    J

    johnsonwax

    Ars Legatus Legionis
    14,407
  • #352
    nimro said:
    Yes PII that fits their automated processing expectations is filtered. If you've ever worked with any amount of raw data you will know that there is zero chance it's all been removed. Did they cover every worldwide format of every possible type of PII? Seems exceedingly unlikely.
    Click to expand...
    At the same time, as someone who has worked with large amounts of raw data, often that I myself collected, there was rarely any benefit in collecting things that didn't match the structure of what I wanted. Just hoovering up random shit isn't helpful unless you are building a search engine (and even that has limits). For anything else you'll burn more resources trying to remove the garbage that your tools can't ingest. There's a robustness principle corollary in there. A list of stolen social security numbers is detrimental to training an AI model. Further, Apple isn't aspiring to build an AGI like OpenAI is, so there's no reason to think that they are taking particularly narrow information. After all, nothing Apple showed indicated that it's capable of, or that Apple wants it to be a tool for providing information. It can't write a paragraph about civil rights because it knows nothing about civil rights, it can only edit one because it does know something about language. That is, there's no specific knowledge baked into the model. Unlike OpenAI you can't ask it for a pizza recipe, but you can ask it for when the pizza place closes, which it'll go out and find out in that moment (which Siri does now). And adding any of that specific knowledge only fucks up the model. What Apple wants is an understanding of language and general information, not an understanding of specific information. Specific information is kind of poison to their effort here because they are trying to make these models work under pretty limited compute budgets and they've wrapped a ton of guardrails around Apple Intelligence to prevent users from even trying to make that sort of thing happen.

    Is there going to be stuff in there that shouldn't be? Sure. Law of large numbers and all that. But I don't see a motive for Apple to want there to be that kind of stuff in there - because everything they have so far shown us prevents the tool from being used in that way. I can't ask Siri to rewrite my book report in nimro's writing style - there's simply no interface to do that. Apple absolutely doesn't want to go down that road.
     
    • Like
    Reactions: Tagbert
    J

    johnsonwax

    Ars Legatus Legionis
    14,407
  • #354
    ev9_tarantula said:
    Doing some back of the envelope calculations I think Google might have somewhere in the region of 1.6 million liquid-cooled TPUs, so 400,000 nodes. On top of that they have air-cooled TPUs for smaller deployments at a wider range of data centres. I'd guess that would double their fleet or more. So it's quite plausible that Google has 4 million TPUs in total, and 1 million nodes*.

    Now I doubt Apple's setup would be as large, but probably at least 10% of that size, and maybe up to half it if they went really all-in on AI.

    * If this sounds ridiculous note that Meta are said to have spent perhaps $30 billion on 1 million Nvidia GPUs, and Microsoft and OpenAI are planning a supercomputer(s) called Stargate that might cost as much as $100 billion and use 5 GW of power.
    Click to expand...
    I mean, Apple has approximately 1 billion TPUs in the wild right now, and about 200 million that Apple Intelligence will run on. That's very clearly the 'datacenter' they expect to be operating on.

    There's also the matter of rapidly diminishing returns in terms of performance. I didn't see anything in Apple's presentation to suggest that the door is open to extremely massive context windows (even though they do compare their server models to GPT-4 Turbo), to doing generative efforts outside of rather narrow scopes, and so on, suggesting that these models are more narrowly tailored to the use cases that Apple provides an interface for. It can summarize an article, but it can't write one. As such, my guess is that AI™ will overperform on-device compared to other models because its utility is narrow and known and controlled by Apple, and that many of the cases where Google and OpenAI need these big data centers (in part because they only operate as a server model) Apple doesn't because there's no interface to do those kinds of things. It's not like AI™ gives you an open prompt to do whatever, so my guess is that the conversational Siri stuff never leaves the device. Some of the larger text processing stuff, particularly on larger works might go to server, and some generative image stuff might, but not like the emoji stuff - the high volume things, summarizing an email chain, etc. Apple's pretty darn good at hitting that Pareto optimal level, even when that leaves them trailing their competition simply due to their scale. It's a recurring exercise for someone to hack Apple features like this onto older hardware, because Apple was conservative in terms of setting user expectations. No reason to believe things will change here.

    Plus, the vast majority of users won't actively use any of this stuff. Adoption curve dynamics apply here as well.
     
    • Like
    Reactions: Tagbert
    W

    wrylachlan

    Ars Legatus Legionis
    12,769
    Subscriptor
  • #355
    johnsonwax said:
    At the same time, as someone who has worked with large amounts of raw data, often that I myself collected, there was rarely any benefit in collecting things that didn't match the structure of what I wanted. Just hoovering up random shit isn't helpful unless you are building a search engine (and even that has limits). For anything else you'll burn more resources trying to remove the garbage that your tools can't ingest. There's a robustness principle corollary in there. A list of stolen social security numbers is detrimental to training an AI model. Further, Apple isn't aspiring to build an AGI like OpenAI is, so there's no reason to think that they are taking particularly narrow information. After all, nothing Apple showed indicated that it's capable of, or that Apple wants it to be a tool for providing information. It can't write a paragraph about civil rights because it knows nothing about civil rights, it can only edit one because it does know something about language. That is, there's no specific knowledge baked into the model. Unlike OpenAI you can't ask it for a pizza recipe, but you can ask it for when the pizza place closes, which it'll go out and find out in that moment (which Siri does now). And adding any of that specific knowledge only fucks up the model. What Apple wants is an understanding of language and general information, not an understanding of specific information. Specific information is kind of poison to their effort here because they are trying to make these models work under pretty limited compute budgets and they've wrapped a ton of guardrails around Apple Intelligence to prevent users from even trying to make that sort of thing happen.

    Is there going to be stuff in there that shouldn't be? Sure. Law of large numbers and all that. But I don't see a motive for Apple to want there to be that kind of stuff in there - because everything they have so far shown us prevents the tool from being used in that way. I can't ask Siri to rewrite my book report in nimro's writing style - there's simply no interface to do that. Apple absolutely doesn't want to go down that road.
    Click to expand...
    I think this is all largely true. But it doesn’t change the fact that scraping the web for training before telling the world they were doing it isn’t a good look relative to their “we’re the good guys” marketing stance. Even if they’re taking extreme precaution it’s not a good look.
     
    Z

    ZnU

    Ars Legatus Legionis
    11,694
  • #358
    johnsonwax said:
    Apple is almost certainly counting on their ability to jam more compute on future processors to reduce the need for server-side compute. They did the same thing with Siri shifting from being entirely cloud-compute to now almost entirely device-compute. So it's a capital cost that they incur to roll the service out which has rapidly declining marginal costs.
    Click to expand...

    It's possible AI capabilities could stall out while efficiency gains and hardware improvements continue, such that AI systems close to the best anyone can build fit on relatively modest hardware. One could argue we're already seeing this, though personally I think it's too early to say. In this case, Apple's datacenter costs should decline, with ~everything eventually moving on-device.

    But I think Apple needs to seriously plan for a world where capabilities keep advancing at the cost of ever-more demand for compute. We really can't rule this out right now, and other companies are making very large bets that suggest they expect it.

    The best human assistants are of above-human-average intelligence, so I don't think there's any natural limit to demand for 'smarter Siri' short of AGI.
     
    W

    wrylachlan

    Ars Legatus Legionis
    12,769
    Subscriptor
  • #359
    ZnU said:
    The best human assistants are of above-human-average intelligence, so I don't think there's any natural limit to demand for 'smarter Siri' short of AGI.
    Click to expand...
    I’m not sure this is meaningfully true. I have a hard time thinking of anything I need in my personal life that I couldn’t get a high school kid to do for me: “Check whether I’ve double booked myself for next weekend” kind of stuff. The idea that ever increasing intelligence will provide ever increasing personal user benefit seems deeply unlikely to me.

    On the work side that may be true, with ever increasing AI power being able to replace more and more jobs. But that’s not the space Apple is playing in, nor are they likely to. On the work side, domain knowledge will probably be key and there will be many players.

    I also think that betting on future advances requiring brute force indefinitely is a bad bet.
     
    Z

    ZnU

    Ars Legatus Legionis
    11,694
  • #360
    wrylachlan said:
    I’m not sure this is meaningfully true. I have a hard time thinking of anything I need in my personal life that I couldn’t get a high school kid to do for me: “Check whether I’ve double booked myself for next weekend” kind of stuff. The idea that ever increasing intelligence will provide ever increasing personal user benefit seems deeply unlikely to me.
    Click to expand...

    Though they do sometimes act otherwise, high school kids possess general intelligence. So this is a pretty minor quibble. We're still talking about a use case that benefits from anything up through actual AGI. And I'd hazard a guess that once you crack AGI, the difference in cost between delivering "average high schooler" performance and "world class executive assistant" performance is pretty negligible.

    wrylachlan said:
    On the work side that may be true, with ever increasing AI power being able to replace more and more jobs. But that’s not the space Apple is playing in, nor are they likely to. On the work side, domain knowledge will probably be key and there will be many players.
    Click to expand...

    Apple doesn't need "Siri, cure cancer" to work, but "Siri, arrange a two week vacation in France" ideally should, someday.

    wrylachlan said:
    I also think that betting on future advances requiring brute force indefinitely is a bad bet.
    Click to expand...

    Well, If everything up through AGI turns out to be cheap, we'll be talking more about Dyson swarms than datacenters.
     
    You must log in or register to reply here.
    Jump to latest Follow Reply
    Jump to latest Follow Reply