Decade since my last build. ML and gaming

Jump to latest Follow Reply
S

ShuggyCoUk

Ars Tribunus Angusticlavius
9,975
Subscriptor++
  • #1
    So I need a new PC. my current one is 11 years old and I now have quite a bit of spare time.

    I am going to use that for two main things:
    • play some games (not that many, but some)
    • Do some (serious) ML upskilling (I have some significant skills and experience in inference, but next to nothing in training) not just LLM, I'm likely to be playing a bit of kaggle.

    I'm no longer able to use someone else's suite of A100's for the latter, and I want to be able to tinker.
    I'm likely to use rental options if I need to when I'm ready, but I want to be able to do some relatively large models locally - so it's a 3090 or a 4090 (24 GB VRAM) or just don't bother. I'm needing a new card anyway so I'm happy to pay a few hundred quid more (and second hand risk) to get the memory.
    I can get a used 3090 for ~£700 (less if I wanted more risk but I can see a nice one right now on GPUsed) and that should do nicely, and cover the gaming.
    I'm not on a 4K monitor, it's 2560x1600 Dell U3014 but it's big enough that pixel pushing is nice - though I am aware that even now the 3090 is overkill, there's just no cheaper way to get the VRAM though.
    That's my gaming target resolution though.

    So based on that I'm looking to build it around, in decreasing order of strength of feeling...

    Vendor: Scan
    - Used them for literally decades. Happy with the quality and service

    OS: Needs to be able to dual boot windows and Linux (likely Ubuntu) If proton works I might consider sticking in Linux though

    GPU: Asus RTX 3090 ROG STRIX
    • Implies I need a power supply with 3 8pin connectors just for the GPU, and realistically at least 750W?
    • theoretically I could get another one and NVLink them for more memory - but I highly doubt it and wouldn't bother building a system to support that later - I mention this for completeness only.

    Case: Fractal North Charcoal black
    • I lean to the mesh, not the glass but happy to be convinced otherwise
    • love their cases generally, like the look and know people very happy with it

    CPU: I lean to AMD
    The L3 cache in particular is lovely when doing CPU inference but I'm not sure on consumer hardware side - this I'm least familiar with because all my recent experience is their server class EPYC stuff. I suspect threadrippers are totally overkill here though for me because for most of the time the GPU will be right there.
    Seems like the 7800X3D gets me reasonable L3 cache and clocks without pushing things out (and keeping the TDP at 120W which is nice). Is that likely to cripple the 3090 in anyway, feels like it shouldn't? 7900 gets me more cores but less cache
    No NPU isn't a big deal - I'm not going down that rabbit hole (yet)
    Going AM4 feels like too backwards looking even based on the increased cost it pushes you down.

    Power: Modular and silent.
    Corsair RM850x seems okay and I think would safely cover the GPU and CPU?

    Motherboard: No clue
    Obviously dependent on the CPU etc, I don't need wifi, nor overclocking capabilities. stable and good value are fine here. Ability to scale to large amounts of memory if I want it would be nice (see below)
    I left this the default on the build page which is ASUS PRIME B650M-A WIFI II - PCIe 4.0, 2x M.2, 2.5GbE/WiFi6 but in theory I could swap out anything scan has. Not sure PCIe 5 would matter at all given the age of the graphics card.

    Memory: no clue
    Again dependent of the CPU/Board. I'm used to having a TiB on hand and just not worrying about it so I lean to making sure I have quite a bit without going nuts. AM5 means more expensive though.
    5600 MHz 2x32 (so I can do 4x later for not too much)

    Cooler: If I buy retail AMD I get a cooler included. given I'm not OC'g I figure stock should do? Quiet above all else here

    Drive: M2 - probably 2 TB, be nice if can also stick in the old 840 EVO 1TB I keep photos etc on if I want
    Scan have a 2TB Solidigm &G read, 6.5 Write for the same price as the Samsung 980 Pro with notionally worse specs. Never heard of them but I see it's basically Hynix selling their own stuff. Or I can bump up a few extra quid for the Corsair MP600 PRO NH

    So here's me configuring it: https://www.scan.co.uk/3xs/shared/c...gshare&utm_medium=link&utm_campaign=copypaste

    Thoughts very welcome, especially the CPU/motherboard/Memory

    Comes to about £1600 + £700 Hitting more like 2K would have been nice (that's about what my builds normally come to and I expect years of service out of them), but that's the 3090 hit I think.
     
    Last edited:
    H

    hobold

    Ars Tribunus Militum
    2,657
  • #2
    The 7800X3D can drive any GPU without bottlenecking, and doesn't care much about memory timings (the huge L3 cache sufficiently decouples the processor from main memory). Going with Ryzen 7000 opens up the (possibly irrelevant) option of using AVX-512 for specialized ML tasks.

    PSU could require a bit of safe guarding; if I recall correctly, the 3090 GPUs could cause unusually high power spikes. Those weren't long enough to be problematic in and of themselves, but would cause some PSUs to trigger shutdowns out of panic.

    I see no reason to pick an AM5 board above B650. PCIE 5.0 won't be a necessary thing in consumer space for quite a while, and the 7800X3D does not tax power delivery at all.

    Linux: Ryzen 7000 has been around long enough that most things should be supported out of the box. Ubuntu is one of the more conservative distros, so the worst that could happen, IMHO, is that you have to manually switch to a newer kernel. Or another distro that is closer to the latest releases.
     
    • Like
    Reactions: ShuggyCoUk
    steelghost

    steelghost

    Ars Praefectus
    4,975
    Subscriptor++
  • #3
    I can't really comment on the CPU / board / RAM side of things as my own main box is still AM4 and will be for a while to come. However on the cooling end of things, according to AMD you don't get a cooler included in the box.

    Given the relatively tame TDP of the 7800X3D, you could go AIO or air. I'd be looking at a beefy dual tower air cooler from Thermalright, Noctua or BeQuiet! - yes probably overkill, but overkill cooling is quiet cooling :D
     
    • Like
    Reactions: pasorrijer and ShuggyCoUk
    S

    ShuggyCoUk

    Ars Tribunus Angusticlavius
    9,975
    Subscriptor++
  • #5
    hobold said:
    The 7800X3D can drive any GPU without bottlenecking, and doesn't care much about memory timings (the huge L3 cache sufficiently decouples the processor from main memory). Going with Ryzen 7000 opens up the (possibly irrelevant) option of using AVX-512 for specialized ML tasks.
    Click to expand...
    Ah yes. that does help in some cases and IIRC ONNX runtime will use it on AMD.

    hobold said:
    PSU could require a bit of safe guarding; if I recall correctly, the 3090 GPUs could cause unusually high power spikes. Those weren't long enough to be problematic in and of themselves, but would cause some PSUs to trigger shutdowns out of panic.
    Click to expand...
    Right, will bump it up - that gives me scope to replace the GPU with another monster in future if I really want to
    hobold said:
    I see no reason to pick an AM5 board above B650. PCIE 5.0 won't be a necessary thing in consumer space for quite a while, and the 7800X3D does not tax power delivery at all.
    Click to expand...
    great, good to know
    hobold said:
    Linux: Ryzen 7000 has been around long enough that most things should be supported out of the box. Ubuntu is one of the more conservative distros, so the worst that could happen, IMHO, is that you have to manually switch to a newer kernel. Or another distro that is closer to the latest releases.
    Click to expand...
    That's good to know thanks. Since the linux environment is basically an ML playground there's a good chance I'll pick specific builds anyway in future
     
    Nevarre

    Nevarre

    Ars Legatus Legionis
    24,110
  • #6
    Concur that a 750w PSU is too light for a 3090 or 4090. 850w bare minimum for safety with the spiky power needs and a kilowatt class is better. Buy high quality--ATX 3.0 standard and "gold or better" ideally. Don't skimp with that 3090 GPU. Ironically the 4090 still has serious PSU requirements but they're less steep than a 3090.

    For the typically moderate price difference between the two, there's some argument for going with the X670 over the B650 if you need more high speed USB or more NVMe devices (the higher number of lanes can translate to more M.2 slots although that's motherboard-dependent.) The single PCIe 5.0 NVMe device option is as noted probably not a make-it-or-break-it option, and I'm in agreement that NVMe 5.0 is not worth the high cost or cooling difficulty compared to already high performing NVMe 4.0 in Q1 2024. The price difference between B650 and X670 may not be huge so it may come down to a motherboard-by-motherboard basis. I would have zero qualms buying B650 if the price and feature set were right but X670 is a little better in terms of expansion.

    The -E Extreme versions, however, are really not worth their price increase. The large number of PCIe 5.0 lanes is still a waste. GPUs can't use them, and that's your core use case. Stick with B650 or X670 and not B650E or X760E (IMHO.)
     
    • Like
    Reactions: ShuggyCoUk
    S

    ShuggyCoUk

    Ars Tribunus Angusticlavius
    9,975
    Subscriptor++
  • #13
    Aeonsim said:
    My one recommendation would have been 2x48GB ram so 96GB for now and the ability to upgrade to 192GB later.
    Click to expand...
    I thought about this, but the board I picked specified max 128GB and I'd seen in previous threads here that going much beyond that seemed pointless for gaming.
    I'm thinking when I miss my pervious monster boxes I'll rent some time on them
     
    • Like
    Reactions: continuum
    S

    ShuggyCoUk

    Ars Tribunus Angusticlavius
    9,975
    Subscriptor++
  • #16
    I just wondered how much of that there was out there. So much of my past world could assume an A100 with 80GB, or was functionally tiny and wouldn't matter (so cared more about MIG). L40S are 48 Gb I note, but I've never heard of anyone that bought them.

    I guess I forget that you can get A100s with 40, and there's still plenty of V100s out there
     
    Last edited:
    W

    wobblytickle

    Ars Praetorian
    596
  • #17
    ShuggyCoUk said:
    I just wondered how much of that there was out there. So much of my past world could assume an A100 with 80GB, or was functionally tiny and wouldn't matter (so cared more about MIG). L40S are 48 Gb I note, but I've never heard of anyone that bought them.

    I guess I forget that you can get A100s with 40, and there's still plenty of V100s out there
    Click to expand...
    I can only speak for what I see with ollama (and a little bit with llama.cpp) and I've not even really started down the RAG/LoRA/vectorDB route yet but yeah in ollama there's a bunch of builds of bigger things highly-quantized and smaller things less-quantized that fit between 24 and 48GB. But yeah, even 24GB lots of fun to be had:


    Code: 
    herodotus ~ $ ollama list
NAME                                    ID              SIZE    MODIFIED
jmorgan/dolphin-mixtral:latest          b9af9337cd76    26 GB   7 weeks ago
alfred:40b-1023-q8_0                    d811415ed9a6    44 GB   6 days ago
bakllava:7b-v1-fp16                     ad44fb2047c1    15 GB   4 days ago
dolphin-mixtral:8x7b-v2.7-q6_K          d86885448d31    38 GB   6 weeks ago
dolphin-phi:2.7b-v2.6-q6_K              4a7162481650    2.3 GB  6 weeks ago
dolphin2.2-mistral:7b-fp16              9b8dd36b01b1    14 GB   3 months ago
falcon:7b                               94c9204f0040    4.2 GB  4 months ago
falcon:7b-instruct                      4280f7257e73    4.2 GB  3 months ago
llama2:13b                              981701930bf3    7.3 GB  4 months ago
llama2:70b                              f60ae38a353b    38 GB   4 months ago
llama2:70b-chat-q4_K_M                  f2ca34dc82b8    41 GB   3 months ago
llama2:7b                               7da22eda89ac    3.8 GB  4 months ago
mistral:7b-instruct-fp16                3c68ce3769cd    14 GB   3 months ago
mistral:7b-text-fp16                    1771c1be9e23    14 GB   3 months ago
mistral-openorca:7b-fp16                c3544390282f    14 GB   6 weeks ago
mistrallite:7b-v0.1-fp16                a6c59d49e2a9    14 GB   3 months ago
mixtral:8x7b-text-v0.1-q6_K             cc78c34333ad    38 GB   7 days ago
nous-hermes:13b-llama2-q8_0             7c6c80693fd9    13 GB   4 days ago
openchat:7b-v3.5-fp16                   8941ffc8ac15    14 GB   3 months ago
openchat:7b-v3.5-q6_K                   ba27b088f027    5.9 GB  3 months ago
openhermes:7b-mistral-v2.5-fp16         579d40574c77    14 GB   6 weeks ago
openhermes2-mistral:7b-fp16             aed1532be570    14 GB   3 months ago
phi:latest                              c651b7a89d73    1.6 GB  7 weeks ago
samantha-mistral:7b-v1.2-text-fp16      463d6b8187c1    14 GB   6 days ago
solar:10.7b-instruct-v1-fp16            4077d1b80ff9    21 GB   4 days ago
solar:10.7b-text-v1-fp16                45ac489b4ee5    21 GB   4 days ago
stable-beluga:7b-q8_0                   6589e954f98e    7.2 GB  4 days ago
vicuna:13b-v1.5-16k-fp16                aa225f209212    26 GB   6 weeks ago
yarn-mistral:7b-128k-fp16               24fcccd3ec98    14 GB   3 months ago
zephyr:7b-beta-fp16                     85a4e7a31862    14 GB   6 weeks ago
     
    • Like
    • Love
    Reactions: continuum and ShuggyCoUk
    I

    IceStorm

    Ars Legatus Legionis
    24,871
    Moderator
  • #20
    The 3090 has independent voltage regulators that do not talk to each other. This results in some cards failing because the card pulls too much power. Buildzoid has fixed a couple Gigabyte Vision cards. His analysis has been that by the time the 3090 has figured out it's pulling too much power, it's too late. It has very slow over current protection circuits that take around one second to kick in.

    The 3090 Ti fixed that deficiency. It has a single voltage regulator that controls all the power stages.

    Used 3090s go for around $900-$1k on eBay. Used 3090 Tis go for around $1000-$1200.

    The 4090 FE pops up from time to time (like yesterday) at MSRP, though nVidia will charge you sales tax if applicable. If you can't wait that long, and you live in a state that both has sales tax and that B&H's Payboo card supports, you can pick up an AIB 4090 for around $1830 right now. Payboo pays you back the sales tax at time of purchase, so the total cost would be just whatever the card price is on the site. That can easily put a card a B&H at only around a $100-$130 price premium over a FE model.
     
    • Like
    Reactions: wobblytickle
    S

    ShuggyCoUk

    Ars Tribunus Angusticlavius
    9,975
    Subscriptor++
  • #22
    Apparently my white RAM was incompatible. But a black set was.

    Since I didn't care what colour it was at all this is not a big deal. But I'm amused at the fact they had to call me to check I was fine with the substitution.

    If the improved radiant heat rejection by going to black was what mattered then I'm going to have a devil of a time with cooling once the 3090is in there :)
     
    S

    ShuggyCoUk

    Ars Tribunus Angusticlavius
    9,975
    Subscriptor++
  • #23
    • Like
    Reactions: steelghost
    You must log in or register to reply here.
    Jump to latest Follow Reply
    Jump to latest Follow Reply