Luckily for everyone, UniSuper had someone store backups in another cloud providers storage.
I don't know how messy this would be if all data was destroyed and there wasn't a way to bring it back.
while (true) {
- Thread starter hanser
- Start date
I don't know how messy this would be if all data was destroyed and there wasn't a way to bring it back.
I have two particularly strong opinions where public cloud is concerned:
1. If you aren't confident you could delete your cloud infrastructure and get it back exactly the way it was, you have work to do
2. If your only copies of source code and backups are in the same cloud accounts as the rest of your stuff, you're doing it wrong
I guess now I have a real-life use case to support both those opinions at once! My god, what a shit show
1. If you aren't confident you could delete your cloud infrastructure and get it back exactly the way it was, you have work to do
2. If your only copies of source code and backups are in the same cloud accounts as the rest of your stuff, you're doing it wrong
I guess now I have a real-life use case to support both those opinions at once! My god, what a shit show
It's nice to know that Google treats everyone the same.
Nah, they don't just up and delete their own services' data. SRE practices apply to things they run for themselves. Those apparently don't apply to customer accounts at all.
Nailed it!
My sister sent me a screenshot today of a convo with her husband. It's a picture of his IDE with a pile of familiar looking code in it and the message "We feel like the comment at the top is Ardax's doing..." ("We" in this case referring to his team at work that I used to be part of a few years back.)
Of course it's something snarky about optimizing for readability rather than performance: "a.k.a. nested loops go brrrrrrrrr....."
Now, I remember the process of writing and testing that code initially and it's pretty hairy. So along with lulz I did also try to comment the entire class fairly well to make the life of anyone who came along later easier. The thing that's tough is that it's not always easy to tell if the comments you write are going to really be helpful or not, because you've got all the context in your head at the moment.
Turned out he was able to follow the code without too much trouble. Felt good.
(As for why he was back in the code: He's writing an integration test that exercises it and needed to know how it worked.)
My sister sent me a screenshot today of a convo with her husband. It's a picture of his IDE with a pile of familiar looking code in it and the message "We feel like the comment at the top is Ardax's doing..." ("We" in this case referring to his team at work that I used to be part of a few years back.)
Of course it's something snarky about optimizing for readability rather than performance: "a.k.a. nested loops go brrrrrrrrr....."
Now, I remember the process of writing and testing that code initially and it's pretty hairy. So along with lulz I did also try to comment the entire class fairly well to make the life of anyone who came along later easier. The thing that's tough is that it's not always easy to tell if the comments you write are going to really be helpful or not, because you've got all the context in your head at the moment.
Turned out he was able to follow the code without too much trouble. Felt good.
(As for why he was back in the code: He's writing an integration test that exercises it and needed to know how it worked.)
I have always liked notmuch. I used it briefly for a period at work some time ago, because we were forced to use OWA and search was very insufficent for me (and Thunderbird didn't help much either). It worke beautifully.
But these days one of my crazy projects is writing my own email client (I'd like to have multiple accounts, a terminal interface, and Markdown composition).
Also, the Python's standard library cmd is "bad", but charming. It's kinda of a good example of the batteries included; it's what
(The main idea behind the client is that it will invoke
But these days one of my crazy projects is writing my own email client (I'd like to have multiple accounts, a terminal interface, and Markdown composition).
notmuch has kinda of a CLI API, where you can run it and make it spew JSON... which really makes implementing a lot of parts of a mail client... a breeze. I've always loathed dealing with emails, but so far it's working pretty well- I have implemented a basic functionality (checking for new email, reading your inbox) in very few hours.Also, the Python's standard library cmd is "bad", but charming. It's kinda of a good example of the batteries included; it's what
pdb uses (I think), so they expose it. But it's quite rudimentary and likely it's never going to improve...(The main idea behind the client is that it will invoke
$EDITOR to compose emails, and then it will convert Markdown- or something similar to Markdown- into both a properly wrapped plain text part, and a pretty HTML part. Writing emails only in plain text sucks for many clients, while I understand many desire to receive text-only email wrapped at 80 characters. Also, writing in Markdown-like formats is great.)Anyone here do much coding in XCode on a Mac? Interested in whether I should get a Macbook Pro or Air for doing dev
If I go with the Pro it's likely purely on wanting the additional memory for VM/ML reasons so if I did that it would be the M3 Max I think
Not bothering summarising the number of cores. I suspect it's largely irrelevant to the coding experience even at the 4+4 on the Air.
The sheer weight and size of the 16" I think rules it out.
Configuring the 15" Air to 24 GB and 2TB is £2,500.
The 14" Pro in the M3 Max gets confusing.
Either I go:
14/30 (cpu cores, gpu cores) then pay £800 extra to go to 96GB for £4,099
16/40and get 64 GB for £3,999 (or another £800 to 128 but that seems nuts for dev)
If things can work well in 24 GB then the Air is the obvious choice, Otherwise it's the 64 GB 14" Pro
Thoughts very welcome
MacBook Air - Compare Models
See what makes each Mac laptop and desktop different. And find the one that’s perfect for your life, your work and your budget.
www.apple.com
If I go with the Pro it's likely purely on wanting the additional memory for VM/ML reasons so if I did that it would be the M3 Max I think
Not bothering summarising the number of cores. I suspect it's largely irrelevant to the coding experience even at the 4+4 on the Air.
| 15" Air | 14" Pro | 16" Pro | |
| Neural Engine | 16 cores. All Same as far as I can tell bar memory bandwidth | ||
| Memory (GB) | 8,16,24 I'd go 24 | 18-128 Likely 64 or 96 | same as 14" Pro |
| Memory Bandwidth (GB/s) | 100 | 150 or 400 (likely 400) | same as 14" Pro |
| Storage (TB) | 0.25-2 | 0.5-8 | 0.5-8 |
| Display (Tech, diagonal in) | Liquid Retina, 15.3in | Mini LED backlit, 14.2 | Mini LED backlit, 16.2 |
| External displays (lid shut) | 2 | ||
| Brightness (nits) | 500 | 600 (HDR much more but I guess that's irrelevant to coding?) | same as 14" Pro |
| Resolution | 2880x1864 | 3024x1964 | 3456x2234 |
| Refresh Rate (Hz) | not sure - I think 60? | up to 120 | Same as 14" Pro |
| Weight (Kg, Pounds) | 1.51, 3.3 | 1.62, 3.6 (M3 Max) | 2.16, 4.8 (M3 Max) |
| Dimensions (HxWxD cm) | 1.15x34.04x23.76 | 1.55x31.26x22.12 | 1.68x35.57x24.81 |
| Thunderbolt/USB 4 | 2 (both on left) | 3 (2 left, one right) | Same as 14" Pro |
| Battery - wireless web most useful I think (Hours) | 15 | 12 | 15 |
The sheer weight and size of the 16" I think rules it out.
Configuring the 15" Air to 24 GB and 2TB is £2,500.
The 14" Pro in the M3 Max gets confusing.
Either I go:
14/30 (cpu cores, gpu cores) then pay £800 extra to go to 96GB for £4,099
16/40and get 64 GB for £3,999 (or another £800 to 128 but that seems nuts for dev)
If things can work well in 24 GB then the Air is the obvious choice, Otherwise it's the 64 GB 14" Pro
Thoughts very welcome
I code on a Mac and on a AMD Linux box. I don't use xcode directly, but CMake + VSCode to keep my stuff portable. Fairly happy with my set up. I don't run with the pre-installed compiler suites, but use homebrew to install the latest version of LLVM. I do code mostly in C++ though. With a side line of python. Homebrew is brilliant for installing all the standard opensource unix bits that don't come with MacOS. I'm quite happy with it all.
I have an unrational dislike of Windows.
I have an unrational dislike of Windows.
macOS handles memory very differently on Intel vs. AS in my experience. RAM isn't as clutch as it used to be in the Apple Silicon world since the memory manager seems to handle it differently. I don't think for an AS machine you need to max it out as much as was necessary in the Intel world.
Given your post and my usual advice to buy as much as you can afford (with the caveat that 128 GB of RAM is bonkers but awesome and almost certainly completely unnecessary), I'd probably recommend the 16/40 with 64 GB*. It's extraordinarily future proofed and well suited to (what I perceive are) your needs.
That said, if you'd rather prioritize lightness/portability over power (or price), the Air will still be one heck of a machine.
* I had a 64 GB M1 for my last job. As a Java dev who runs IntelliJ IDEs I'm a bit of a RAM user, and I found the memory usage almost never got above 32 GB even when I was running multiple servers and IDE instances. I could force it way high by giving IDEA some crazy memory settings, but Apple seems to have really upped their memory management game in AS Macs. I'm fairly confident that 64 GB will suffice for the useful life of the machine unless you're already working with extraordinarily large memory requirements.
Given your post and my usual advice to buy as much as you can afford (with the caveat that 128 GB of RAM is bonkers but awesome and almost certainly completely unnecessary), I'd probably recommend the 16/40 with 64 GB*. It's extraordinarily future proofed and well suited to (what I perceive are) your needs.
That said, if you'd rather prioritize lightness/portability over power (or price), the Air will still be one heck of a machine.
* I had a 64 GB M1 for my last job. As a Java dev who runs IntelliJ IDEs I'm a bit of a RAM user, and I found the memory usage almost never got above 32 GB even when I was running multiple servers and IDE instances. I could force it way high by giving IDEA some crazy memory settings, but Apple seems to have really upped their memory management game in AS Macs. I'm fairly confident that 64 GB will suffice for the useful life of the machine unless you're already working with extraordinarily large memory requirements.
I have a 64 Gb M1 Max Mac Studio and a 32 GB M1 Max MBP and I endorse this post. I usually run several VMs simultaneously plus a full complement of other tools, including JetBrains IDEs. 32 GB is enough, 64 is not going to be short for anything other than very, very large datasets.
I've got a 14" M2 Max MacBook Pro, with 96 GB of RAM. My thinking was that I want to not have to worry about the footprint of multiple VMs or heavyweight scientific models while trying to do other things concurrently.
I have a 14" M3 MacBook Pro with 34GB RAM.
I usually have about 8GB-10GB free at most times (Chrome, Firefox, IntelliJ, Sublime Text, etc). The only times I get really low on memory is when I spin up a bunch of containers in Docker when playing with k8s stuff.
Only thing I regret is sticking with the default 500GB SSD instead of 1TB.
I usually have about 8GB-10GB free at most times (Chrome, Firefox, IntelliJ, Sublime Text, etc). The only times I get really low on memory is when I spin up a bunch of containers in Docker when playing with k8s stuff.
Only thing I regret is sticking with the default 500GB SSD instead of 1TB.
I have 36 GB on my work machine (M3 Max Macbook Pro); I have yet to bring it close to swapping during normal work. (The one time I did was by accident-- misconfigured something and ended up with combinatorial explosion trying to throw tens of trillions of points into RAM instead of the few thousand I'd intended. No Mac would hold the petabyte of RAM needed for that particular bit of code to have actually completed.)
Part of that is because my main use-case for VMs (Windows) kind of went away when I switched over to ARM-- my Windows VMs have to be Intel, so that work moved over to separate physical machines. And any Linux VMs I spin up tend to be small.
Part of that is because my main use-case for VMs (Windows) kind of went away when I switched over to ARM-- my Windows VMs have to be Intel, so that work moved over to separate physical machines. And any Linux VMs I spin up tend to be small.
I have llama3 running under ollama right now; it's sitting at about 300 MB of memory usage per Activity Monitor, but I'm not convinced that it's accounting for everything that's actually allocated to it (is the model itself a memory-mapped file or something that's not being listed as memory usage for the ollama server? The whole thing has to be in RAM somewhere, and it's a 4.7 GB download.).
Regardless, with that plus all my stuff for work (VS code, Slack, Chrome, Mail, all that fun stuff), I'm sitting at ~22.8 GB used of 36.
Is mobility a concern?
If I were to get a new laptop I'd want it to have a petabyte of RAM and a 30' screen, of course. But the thing is, my current MBP is in battlestation mode 95% of the time, whether I'm working from home or at the office. Plugged in, external monitor(s) connected, external keyboard and mouse likewise. It's been so long since I've used it on a train or something, I'd probably cringe doing it.
If mobility / battery life matters to you, consider the Air. Otherwise, one of the Pros.
(Also, the more beefy the laptop is the more likely it is to handle the shit software well. You know, the software you don't want, but have to use for practical reasons. In my case it's Microsoft Teams. Takes 30s to start and frequently slows down my work enough that I have to log in to the iOS version instead. Hate that thing.)
I'm not sure yet. but the difference from the Air to the 14"Pro is negligible enough. The 16" is just a monster, I think if I'm using that as a base I'll just get a Studio and be done with it
I can’t recommend the current crop of 14’’ mbp enough. A BTO M3 Max with 128gigs of RAM will last a long time.
I was tempted to go for a studio and carry it with me when travelling, but having used mac laptops for now two decades, the quality of the current 14 mbp is astounding.
I would love for a redesign with the current keyboard and the touch bar on top of the F-keys, but that’s just wishful thinking.
For ML - I still believe that a Linux box with a 4090 or EC2 GPU instances are better value when training models.
When finances permit, I’ll trade my 2018 i9 16’’ for a smaller, faster m3 max with joy.
I was tempted to go for a studio and carry it with me when travelling, but having used mac laptops for now two decades, the quality of the current 14 mbp is astounding.
I would love for a redesign with the current keyboard and the touch bar on top of the F-keys, but that’s just wishful thinking.
For ML - I still believe that a Linux box with a 4090 or EC2 GPU instances are better value when training models.
When finances permit, I’ll trade my 2018 i9 16’’ for a smaller, faster m3 max with joy.
128 seems nuts, but it would let a hefty model stay resident whilst other things run.
This would be inference, not training. Depends if I use it for work or not I need to find that out.
OMG THIS SO FUCKING MUCH. THERE'S ROOM FOR BOTH!
I'm glad I'm not the only one who thinks it was a mistake for Apple to throw that away. I mean, it wasn't perfect. Adding the haptic feedback from Better Touch Tool makes it 1000x better. But if Apple had actually developed it I think it could have been a really good tool.
Yeah, I run Llama models on my 64 GB Mac Studio to play with them, using llama.cpp. Quantized 70B parameter models are no problem; they take about 36 GB of RAM. Performance is... acceptable (it's 'only' an M1 Max). Enough to play with, not enough that I'd like to use it production anywhere.
Is every configuration of GIT dog slow? Over the years, I've used file system-based remotes (slow), and network server-based remotes (slow). I've tried GitHub remotes (slow), and Atlassian Stash remotes (slow). For years I thought "Maybe when I get a faster network connection, or maybe when I get a faster machine" But fetches, pulls, commits, and pushes always take way longer than I expect.
I understand that diffing files takes lots of processing power, but Git has always seemed excessively slow to me.
Are there any configurations of GIT and GIT remotes that are snappy? Or should I get off my grumpy-old-man high horse and stop complaining because syncing a local branch with a remote branch actually should take minutes rather than seconds?
I understand that diffing files takes lots of processing power, but Git has always seemed excessively slow to me.
Are there any configurations of GIT and GIT remotes that are snappy? Or should I get off my grumpy-old-man high horse and stop complaining because syncing a local branch with a remote branch actually should take minutes rather than seconds?
I suspect a lot of them run in docker containers and have to share a cpu with 73 other git VM instances.
If you're seeing pulls that routinely take multiple minutes with Github or Bitbucket, something's not right on your end-- maybe an excessively large repository (do you have large binary files that frequently change in your Git repository?). Or you're on ancient hardware and your local machine (or local disk) is too slow.
git pull should be more on the order of tens of seconds (or less) unless you're Google and running your entire business out of a milions-of-lines-of-code monorepo.Before blaming the implementation of git I would definitely look at what's in the repo.
One place I worked had a slow (and very big) git repo. It was a bad repo to deal with because they didn't think about git's limitations and just converted their old Subversion repo over to git. The repo had hundreds of jpegs (it was a website) and had big Photoshop and Quicktime videos in their history.
Google did a lot of stupid shit with source control because they had a shit ton of money and smart people to throw at the problem instead of actually thinking up a smart and sane solution.
At one point, when they were still on Perforce and insisting on a gigantic monorepo, it would take 20+ minutes for a commit to go through. They had a team who monitored the Perforce server(s) and would kill off problematic commits and then reach out to the person on how the commit should be crafted to avoid excessive commit time. This is from when I interviewed with them around almost 2 decades ago.
Code:
$ time git clone https://github.com/django/django.git
Cloning into 'django'...
remote: Enumerating objects: 528200, done.
remote: Counting objects: 100% (576/576), done.
remote: Compressing objects: 100% (382/382), done.
remote: Total 528200 (delta 317), reused 341 (delta 190), pack-reused 527624
Receiving objects: 100% (528200/528200), 247.34 MiB | 12.85 MiB/s, done.
Resolving deltas: 100% (386883/386883), done.
real 0m36.148s
user 0m48.299s
sys 0m4.940s
alex@molly:~/tmp$ cd django/
$ git reset --hard $(git rev-list -n1 --before=2024-01-01 HEAD)
HEAD is now at c72001644f Updated DatabaseFeatures.bare_select_suffix on Oracle 23c.
alex@molly:~/tmp/django$ time git pull
...
real 0m1.015s
user 0m0.433s
sys 0m0.127s
$ time git fetch
real 0m0.593s
user 0m0.049s
sys 0m0.036sThis is in a computer where Slack or any other Electron app is painful.
git clone can be slow- it's proportional to the size of the repo (Django's repo is 263mb- it's not huge, but it's very likely larger than any of my personal projects). Any other operation in most of the repos I work with is pretty fast.(Roughly, to check the size of your repo,
du -h .git. Compare that to the Django repo 263mb size. That's a nearly 20-years old repo, with a lot of contributors and a not-small code base.)Uhh, no. One big repo I work with has 5M LoC and its
.git directory is 2.8 GB in size (on my machine), and pulls and pushes take on the order of seconds. If it's taking more than 10s then a) I start cussing, and b) I check my internet connection.Maybe give us some more information on your repo and its size and contents?
Git (not GIT) was designed and gained appeal in part on having data structures and protocols that made it much faster than contemporary alternatives, as well as being more capable.
If it's not living up to that, either your client system has issues, or you're using it in some very unusual or inadvisable fashion.
In terms of operations happening, there is no file diffing occurring during any remote repo operations. All diffing happens locally, as part of creating a commit, or actually comparing things. Push and pull operations transfer those commit objects in terms of binary blobs that represent the minimum of content that the recipient side doesn't already have.
Last edited:
Do git fetch rather than pull and see if that’s the problem rather than the merge locally part.
If you have binaries/zip in your repo then the fetch might be triggering anti virus checking
If you have binaries/zip in your repo then the fetch might be triggering anti virus checking
Also check if a GC is triggering on almost every operation, we had that problem on windows (where the difference in the file system approach makes that more expensive) and massively increasing the GC threshold was a big win.
Git - git-gc Documentation
See configuration.
gc.auto=0
Completely stops the auto doing anything, it’s a very blunt tool though so try it and see first. There are no end of tweaks there that might work better, but I can’t recall the exact set we used (and can no longer check, sorry)
On Windows IIRC the auto detach doesn’t work, which is the other approach to solve it
So... at $NEW_JOB, we have a project where the complexity of the build/develop/run/package process might be bigger than the software itself.
Well, it's software quite embedded into the Android OS guts. For efficiency, part of the build is distributed (the Android build system requires 32gb of RAM, still we want to edit locally, so we build on bare metal beefy servers). One functionality has complex networking requirements, so we need to support running the software with Docker, to use virtual networks. Plus, running an Android emulator conveniently is moderately complex. In addition to that, we need quite a bit of automation for efficiency. And of course, testing everything with GitHub Actions is complex (and on top of that, we need self-hosted runners).
I'm a bit bothered because I think it's all "essential" complexity, not really "accidental"- we've managed to remove a good chunk of accidental complexity from the scripts, but I kinda wonder if I'm missing the forest for the trees and we're really overengineering. I can't notice much overengineering... but you frequently don't notice it! We always think it's essential complexity, even when it's accidental...
(My other thought is that perhaps we ought to do consulting/SaaS of builds for the industry we are targeting. That might be more valuable than the software itself we develop. I think we could build pretty nice cloud IDEs which would provide a ton of value to companies in the industry.)
Well, it's software quite embedded into the Android OS guts. For efficiency, part of the build is distributed (the Android build system requires 32gb of RAM, still we want to edit locally, so we build on bare metal beefy servers). One functionality has complex networking requirements, so we need to support running the software with Docker, to use virtual networks. Plus, running an Android emulator conveniently is moderately complex. In addition to that, we need quite a bit of automation for efficiency. And of course, testing everything with GitHub Actions is complex (and on top of that, we need self-hosted runners).
I'm a bit bothered because I think it's all "essential" complexity, not really "accidental"- we've managed to remove a good chunk of accidental complexity from the scripts, but I kinda wonder if I'm missing the forest for the trees and we're really overengineering. I can't notice much overengineering... but you frequently don't notice it! We always think it's essential complexity, even when it's accidental...
(My other thought is that perhaps we ought to do consulting/SaaS of builds for the industry we are targeting. That might be more valuable than the software itself we develop. I think we could build pretty nice cloud IDEs which would provide a ton of value to companies in the industry.)
One small simplification possibility catches my eye is that encapsulating the builds in Docker containers could let you manage the environment running that the same as (or together with) the testing environment with the simulated networking. Bare metal servers with real workloads are a pain to manage in various ways, and ideally would be mostly an abstraction of where containerized jobs get distributed to.
You might be able to consolidate some or all of the infrastructure through some Infrastructure as Code approaches, using stuff like OpenStack, Kubernetes, and OpenTofu. This is the practical extent of my knowledge of those sorts of tools, though.