Java & Spring Boot - hamstringed by server resources?

meanderbot · Apr 16, 2024

Hello, newbie web master (and forum poster) here. Just looking for confirmation about a theory I have:

I'm making my first attempt at implementing Spring Boot services on a web host. On my machine, with a bare minimum Spring Boot project (with just Spring Web) runs fine but trying to run it on my web host I get several "unable to create thread" exceptions, with hints that it is related to a lack of resources.

The website was just supposed to be a personal site to practice this sort of stuff; I wasn't expecting any great bandwidth or resource issues so I went with a pretty cheap plan. I see that I'm limited to about 50 threads and/or processes, which I believe I'm maxing out every time I attempt to launch the Spring Boot jar file.

So it seems to me that I just need to upgrade for more resources. Does that sound right?

Also, if anyone has suggestions for hosts or a guideline for specs I might need for (I'm assuming) simple Spring Data stuff like retrieving/storing blog posts, I'd love to hear them. TIA

Edit: it's since occurred to me that I'd probably be better off just using a cloud compute service to host my Spring app. Thoughts?

koala · Apr 17, 2024

It sounds like you're using one of those classic "shared hosting" services that were the rage for hosting PHP back in the day? They would give you an FTP to upload files and a MySQL database?

(These are actually the grandparents of some of today's fancier hosting services, so they were actually quite innovative.)

Those shared hostings were typically quite anemic, and I guess they're still anemic today. Likely you can tune your application to fit there, most Java stuff defaults to very ample defaults- 40 threads for web serving sounds like a default Spring Boot would use. You're likely hitting RAM constraints too. I have started Spring applications with less than 100mb of RAM, so maybe you could tune things to fit. If it's a simple application, even though Java has a reputation, it's likely you can reduce drastically the amount of allocated resources to it and still run it well.

However, I'm not sure it's even possible to get a VPS today with less than 1Gb, which should make all efforts to tune unnecessary.

You could also look at the modern equivalents of shared hosting nowadays. I did some experiments with render.com a few days ago. This would make it simpler to deploy your application- you wouldn't have to care about OS maintenance, for example.

dio82 · Thursday at 12:44 AM

I am right now scratching my head ...

Java threads are virtual threads executed within the JVM. The JVM itself runs with usually only one thread or a handful of threads depending on "hardware" (most of the time we are talking about docker images, right?) and settings.

Plain Java is always single threaded, but Tomcat as well as Springboot use virtual JVM threads heavily under the hood.

Java servlets will spin up as many threads (remember, in Spring MVC one request= one Java thread) as there is memory available and CPU available for context switching between threads.

So your issue sounds like you are running out of memory. But usually in that case the error is more verbose and tells you exactly that.

You can reduce the memory footprint enormously by pre-compiling the code beforehand to GraalVM, but that is definitely not a beginner topic.

If you just wanna spin up some Java Springboot on the interwebs, get a free tier micro VM with Ubuntu on your preferred cloud vendor and have fun.

Apteris · Thursday at 2:52 PM

dio82 said:
Java threads are virtual threads executed within the JVM.

Not really. The "primary", meat-and-potatoes type of thread used by the JVM (called a "platform thread" as of late) is a native thread, i.e. one which is scheduled by the OS scheduler rather than somehow else. The JVM uses native methods to create threads, and maps them to OS threads.

Very early versions of Java (before 1.3, used around the turn of the millennium) used something called green threads, which were virtual, and which got abandoned.

Very recent versions of Java (21+, though the feature has been baking for a while) use something called virtual threads, which, again, are virtual. But you have to opt in to using them, and you'd better make sure all of your libraries and stuff work fine with them.

dio82 said:
The JVM itself runs with usually only one thread or a handful of threads depending on "hardware"

You might be surprised.

Debugging a Hello World program run on Java 21 I see:

a "main" thread;
an "Attach Listener" thread;
a "Common-Cleaner" thread;
a "Finalizer" thread;
a "Notification Thread" thread;
a "Reference Handler" thread;
and a "Signal Dispatcher" thread.

dio82 said:
(most of the time we are talking about docker images, right?) and settings.

Could be Docker images, could be something else. Depends on how sophisticated that web host provider is.

dio82 said:
Plain Java is always single threaded, but Tomcat as well as Springboot use virtual JVM threads heavily under the hood.

Only quite recent versions of Tomcat and Spring Boot. They'd been using platform threads up until recently.

dio82 said:
So your issue sounds like you are running out of memory. But usually in that case the error is more verbose and tells you exactly that.

I also think this is what's happening. OP can post more details or pass some debug flags to java.exe if assistance is really needed.

dio82 said:
If you just wanna spin up some Java Springboot on the interwebs, get a free tier micro VM with Ubuntu on your preferred cloud vendor and have fun.

I'd use whichever Linux distribution is the smallest and fastest one, preferably the one offered by default by one of the big cloud providers.

Apteris · Thursday at 2:54 PM

meanderbot said:
Edit: it's since occurred to me that I'd probably be better off just using a cloud compute service to host my Spring app. Thoughts?

It sounds like your current web host isn't offering you all of the monitoring features one might wish for.

In any event, I'd just learn AWS if I were you. They have a free tier and it looks good on your résumé.

ramases · Thursday at 4:39 PM

Though on the note of virtual threads, using libraries that are not designed for it (like older versions of Jackson) absolutely can blow up your memory footprint on a per-request basis, specifically when you have a library stuffing a lot of things into ThreadLocals (because it thinks threads are pooled objects of which few exist snd they get recycled, instead of the type of ephmeral single-use threads virtual threads are) and your virtual threads stick around long enough.

The ecosystem right now is IMHO in a bit of a bad spot wrt virtual threads. The primary benefit is so you can use easy, imperative programming to get the same level of concurrency and non-blocking throughput concepts like reactive programming gives you.

But honestly anyone capable enough to figure out if a given stack is capable of fully utilizing virtual threads, can use it without blowing up your memory but will see limited/no benefits (for example because right now using the synchronized keyword will still pin the virtual thread to a platform thread, or because you arecpu bound anyway), or will blow up your memory likely is also capable of using some of the more accessible reactive libraries like Mutiny.

Apteris · Friday at 1:49 AM

ramases said:
anyone capable enough to figure out if a given stack is capable of fully utilizing virtual threads [..] likely is also capable of using some of the more accessible reactive libraries like Mutiny.

My hope is that Virtual Threads prove easier to learn and gentler to debug in a real-world application than did reactive programming. I've studied the documentation and implementation of Project Reactor a fair bit, it has always struck me as being written by wizards from space.

I didn't know about Mutiny, I'll give that a look, cheers.

Lt_Storm · Friday at 3:28 AM

Apteris said:
My hope is that Virtual Threads prove easier to learn and gentler to debug in a real-world application than did reactive programming. I've studied the documentation and implementation of Project Reactor a fair bit, it has always struck me as being written by wizards from space.

I didn't know about Mutiny, I'll give that a look, cheers.

Really? That stuff always struck me as a simple enough concept: Streams are just the dual of Lists. Rather than your choice getting things from an iterator, you turn that interaction upside down by subscribing your code to it and it gives you the next value. Then essentially the same thing happens behind the scenes, at least once the values are available.

On the other hand, I suppose I am the guy who thinks functions (edit: or, more accurately, closures) are just objects turned inside out... So I may not be the best judge of what's space magic vs terrestrial science.

ShuggyCoUk · Friday at 3:58 AM

Mutable state by default and a shared flat memory space plus any mechanism to “hide” real threads is just plain hard to reason about. Doesn’t matter what it is.

Plain threads are also hard to reason about too, but are so in your face it can at least make it undesirable to footgun yourself.

You need to learn the restrictions within that world and IME most people don’t do well early on and get burnt so they move away.

ramases · Friday at 4:47 AM

Apteris said:
My hope is that Virtual Threads prove easier to learn and gentler to debug in a real-world application than did reactive programming. I've studied the documentation and implementation of Project Reactor a fair bit, it has always struck me as being written by wizards from space.

I didn't know about Mutiny, I'll give that a look, cheers.

As far as the Java ecosystem is concerned it probably is the easiest option to get into reactive programming, because it expresses it purely as event-stream processing.

You create event-sources, and those event sources have two types of observers you attach to them:
1) Event Observers: Observe individual events emitted by the event source
2) Source Observers: Observe events about the state of the event source; for example if you have an async pipeline processing N items and want to observe when all items have completed successfully, you observe the onCompletion event.

The following is a good explanation of this pattern: https://smallrye.io/smallrye-mutiny...t-makes-mutiny-different/#an-event-driven-api

Here's also a good primer on Mutiny by Quarkus (and how Mutiny integrates into Quarkus; maybe have a look at it to escape Spring Hell while you are at it?

)https://quarkus.io/guides/mutiny-primer

Apteris · Friday at 2:40 PM

Lt_Storm said:
Really? That stuff always struck me as a simple enough concept: Streams are just the dual of Lists. Rather than your choice getting things from an iterator, you turn that interaction upside down by subscribing your code to it and it gives you the next value. Then essentially the same thing happens behind the scenes, at least once the values are available.

Yeah. The more challenging part (for me at least) is understanding the implementation, something which I need to do both as part of learning something, and when debugging.

Let's consider three ways of doing the same thing, in what I consider to be increasing order of complexity of the behind-the-scenes implementation.

Java:

List<Integer> filterOddNumbersClassic(List<Integer> integers) {

    List<Integer> result = new ArrayList<>();
    for (int i : integers) {
        if (i % 2 == 0) {
            result.add(i);
        }
    }

    return result;
}

Easy. The ArrayList constructor sets just one field, and the add method grows the list if necessary, and then appends the argument. Life is good.

Java:

List<Integer> filterOddNumbersStreams(List<Integer> integers) {

    return integers
            .stream()
            .filter(i -> i % 2 == 0)
            .toList();
}

.stream() calls StreamSupport.stream(spliterator(), false), and that .stream() method's implementation is interesting:

Java:

public static <T> Stream<T> stream(Spliterator<T> spliterator, boolean parallel) {
    Objects.requireNonNull(spliterator);
    return new ReferencePipeline.Head<>(spliterator,
                                        StreamOpFlag.fromCharacteristics(spliterator),
                                        parallel);
}

And the .filter()? Well, the implementation in ReferencePipeline is:

Java:

@Override
public final Stream<P_OUT> filter(Predicate<? super P_OUT> predicate) {
    Objects.requireNonNull(predicate);
    return new StatelessOp<P_OUT, P_OUT>(this, StreamShape.REFERENCE,
                                 StreamOpFlag.NOT_SIZED) {
        @Override
        Sink<P_OUT> opWrapSink(int flags, Sink<P_OUT> sink) {
            return new Sink.ChainedReference<P_OUT, P_OUT>(sink) {
                @Override
                public void begin(long size) {
                    downstream.begin(-1);
                }

                @Override
                public void accept(P_OUT u) {
                    if (predicate.test(u))
                        downstream.accept(u);
                }
            };
        }
    };
}

And from here one would need to read up on what a Sink is, what a ChainedReference is, and so on. So the amount of complexity behind that nice Java streams pipeline is already non-trivial.

Last implementation:

Java:

List<Integer> filterOddNumbersReactive(List<Integer> integers) {

    return Flux
            .fromIterable(integers)
            .filter(i -> i % 2 == 0)
            .collectList()
            .block();
}

(Yes, don't use .block() in Prod, I know.)

I'm not going to paste the implementation code, there's too much of it. I find the code in question both intricate and very "action at a distance-y". Anyone who's debugged this sort of thing knows the sort of stack trace you get, and how tricky it is to pinpoint when exactly something happened.

---

All in all, I'm not complaining. This stuff is interesting and it pays well. But there are cost-of-complexity - benefits considerations.

Lt_Storm · Friday at 3:23 PM

Apteris said:
I'm not going to paste the implementation code, there's too much of it. I find the code in question both intricate and very "action at a distance-y". Anyone who's debugged this sort of thing knows the sort of stack trace you get, and how tricky it is to pinpoint when exactly something happened.

---

All in all, I'm not complaining. This stuff is interesting and it pays well. But there are cost-of-complexity - benefits considerations.

I suppose I get what you mean... Though describing it as space wizards having written it still strikes me as nuts. After all, any real space wizard would have looked at one of those stack traces, been offended, and sat down to write some kind of tool to summarize the stack traces succinctly. Perhaps it was written by a space wizard's apprentices?

Apteris · Saturday at 1:38 AM

Lt_Storm said:
any real space wizard would have looked at one of those stack traces, been offended, and sat down to write some kind of tool to summarize the stack traces succinctly

They have. See for instance Debugging Reactor: the project offers several ways to capture interesting events and correlate them, in among the long, asynchronous pipelines being called. There is a "rewrite my stack trace to make it more intelligible" feature.

In any case, back to studying.

Mark086 · 2024-06-30T10:17:47-0400

Cheap hosting with the freedom to do what you want. This is not necessarily "production" ready, although it can be.

Set up a machine with Linux, set up docker or podman. Configure your web server stack, and configure a Cloudflare Tunnel (they have a container for this, use theirs).

This will get you tunnable instances of your tools, size them to fit, adjust accordingly.

Don't forget to create a backup process.

I have a raspberry pi sitting on a shelf running a web stack for a family website, the Rpi4 is running with a 500 Gig SSD, configured to boot directly off it, and it has been running like a dream since I set it up.

You don't have to use a Pi, but this is the cheapest hosting you can get, with the most control over the server and can work excellent to size out a system without taking too many risks on a VM with a cloud provider.

Create the docker containers to work properly and the process is repeatable on any cloud service with VM or container services.

I don't experiment on cloud VM as much because of this. I do my experiments on my server and then decide what to do with it. In my case I started doing this to gain experience with docker & podman because of my day job, but the combination of tools + cloudflare tunneling makes this viable for small sites with unique requirements, prototyping etc.

Cloudflare isn't the only option, but they do provide a container and the tunneling is free at the low end. I was looking at Tailscale as well, but I didn't proceed with them due to success with Cloudflare.

This doesn't solve your Java issues; it does give you full control over a server with no usage costs.

Java & Spring Boot - hamstringed by server resources?

Smack-Fu Master, in training

Ars Tribunus Angusticlavius

Ars Tribunus Angusticlavius

Ars Tribunus Angusticlavius

Ars Tribunus Angusticlavius

Ars Tribunus Angusticlavius

Ars Tribunus Angusticlavius

Ars Praefectus

Ars Tribunus Angusticlavius

Ars Tribunus Angusticlavius

Ars Tribunus Angusticlavius

Ars Praefectus

Ars Tribunus Angusticlavius

Ars Tribunus Angusticlavius