How far do you see lemmy.world capable of scaling to? One thing I’ve been noticing is the centralisation of Lemmy users on a few top servers, surely that cannot be healthy for federation? What are your thoughts on this?
Not entirely sure of what you are asking, but the only reason they need a clustered setup is simply because of their scale. Making the details of their setup public does not help with the issue I addressed, since in an ideal scenario, communities and users would be evenly distributed amongst the many Lemmy instances in the fediverse, making the need to do any sort of clustering for performance reasons unnecessary.
We do run on 1 server, but we’ve now seen that Lemmy scales horizontally so the k8s path forward is open 😊 With all these latest improvements we can have a bit more users on the current box.
Not trying to be pedantic, but why do they have to do so? Why can’t people figure it out themselves? Also, why can’t Lemmy instances run on single non-redundant boxes? Most instance operators don’t have the budget of enterprises, so why would they have to run their Lemmy’s like enterprises?
Er, because we should all be working together to try to help Lemmy grow and be stable…?
I agree with this point, but I disagree with the context in which you mentioned, “They should post their clustered setup so others can replicate more easily”, right as a reply to my original comment asking how Ruud felt about the centralisation of users in a federated application. This should’ve been an entirely separate reply, or perhaps an issue on GitHub to the Lemmy authors.
You can run on a single box, but a single problem will bring down your single box. This is a basic problem commonly discussed in DevOps circles.
Again, I agree, but the context in which you mentioned it, basically suggests that everyone who runs single instance Lemmys are doing it wrong, which I disagree.
Lowering the entry requirements is part of how we can get wide-spread adoption of federated software. Not telling people that they have to have at least 2 instances with redundancies or they are doing it entirely wrong.
The bare minimum I would ask anyone running their own instance, is to have backups. They don’t need fancy load-balancers, or slaved Postgres database setups, or even multi-node redis caches for their instances of sub-thousand users.
For example, one reasonably priced server on most providers is like $20-40/month. Say a load balancer as a service is another $10-20, and a database server or database as a service is also like $20-$40. A distributed, redundant setup would be like 2 webservers, a database, and a load balancer so like, $70?
Seriously? That may be an acceptable price tag for a extremely public Lemmy host, like lemmy.world or lemmy.ml, but in no way should it be a reasonable price tag for the vast majority of Lemmy instances setup out there. Especially when most of them have sub-thousand users. $70/mo? That has to be a joke. You can easily host a Lemmy on a $5-$10 droplet for ~100 users.
I’ve deployed clustered applications myself, I just haven’t looked into doing it with Lemmy and was curious if they had a run book or documentation.
No offense, but you definitely seem like the kind of person to shill for cloud-scaling and disregard cost-savings.
The point of federation means that even an instance with just a handful of users, can participate in larger communities on other servers that may have thousands of users. From that thousands of users, may also include handful of users FROM other instances. That’s the beauty of federation. While a community needs to be on one instance, the people do not.
I’m on a server with <50 users, and yet I’m still subscribed to the larger communities, and enjoying in delightful conversations with people such as yourself. Why am I on server with <50 users? Simply because it performs better, and through the magic of federation, doesn’t require me to have to register to a server halfway around the globe just because everyone else is there too.
I have no idea what led you to believe that there’s no use-cases for smaller instances, and I would definitely like to hear the arguments behind it.
Again, for the purposes of smaller Lemmy instances (< thousand users), there is hardly a requirement for any kind of load balancing. Having one, is my definition and rationale for calling it fancy. It’s over-engineering a solution that is not a problem for majority of instances. Your arguments thus far would apply to larger Lemmy instances, even Reddit-lite, but that’s not what federation is about. It’s not a bunch of large instances forming a fenced community and gatekeeping the tiny ones out.
How far do you see lemmy.world capable of scaling to? One thing I’ve been noticing is the centralisation of Lemmy users on a few top servers, surely that cannot be healthy for federation? What are your thoughts on this?
deleted by creator
Not entirely sure of what you are asking, but the only reason they need a clustered setup is simply because of their scale. Making the details of their setup public does not help with the issue I addressed, since in an ideal scenario, communities and users would be evenly distributed amongst the many Lemmy instances in the fediverse, making the need to do any sort of clustering for performance reasons unnecessary.
deleted by creator
We do run on 1 server, but we’ve now seen that Lemmy scales horizontally so the k8s path forward is open 😊 With all these latest improvements we can have a bit more users on the current box.
Oh, I could a swore I read somewhere you went multi. Maybe I’m confusing another instance
Not trying to be pedantic, but why do they have to do so? Why can’t people figure it out themselves? Also, why can’t Lemmy instances run on single non-redundant boxes? Most instance operators don’t have the budget of enterprises, so why would they have to run their Lemmy’s like enterprises?
deleted by creator
I agree with this point, but I disagree with the context in which you mentioned, “They should post their clustered setup so others can replicate more easily”, right as a reply to my original comment asking how Ruud felt about the centralisation of users in a federated application. This should’ve been an entirely separate reply, or perhaps an issue on GitHub to the Lemmy authors.
Again, I agree, but the context in which you mentioned it, basically suggests that everyone who runs single instance Lemmys are doing it wrong, which I disagree.
Lowering the entry requirements is part of how we can get wide-spread adoption of federated software. Not telling people that they have to have at least 2 instances with redundancies or they are doing it entirely wrong.
The bare minimum I would ask anyone running their own instance, is to have backups. They don’t need fancy load-balancers, or slaved Postgres database setups, or even multi-node redis caches for their instances of sub-thousand users.
Seriously? That may be an acceptable price tag for a extremely public Lemmy host, like lemmy.world or lemmy.ml, but in no way should it be a reasonable price tag for the vast majority of Lemmy instances setup out there. Especially when most of them have sub-thousand users. $70/mo? That has to be a joke. You can easily host a Lemmy on a $5-$10 droplet for ~100 users.
No offense, but you definitely seem like the kind of person to shill for cloud-scaling and disregard cost-savings.
deleted by creator
That is such a take.
The point of federation means that even an instance with just a handful of users, can participate in larger communities on other servers that may have thousands of users. From that thousands of users, may also include handful of users FROM other instances. That’s the beauty of federation. While a community needs to be on one instance, the people do not.
I’m on a server with <50 users, and yet I’m still subscribed to the larger communities, and enjoying in delightful conversations with people such as yourself. Why am I on server with <50 users? Simply because it performs better, and through the magic of federation, doesn’t require me to have to register to a server halfway around the globe just because everyone else is there too.
I have no idea what led you to believe that there’s no use-cases for smaller instances, and I would definitely like to hear the arguments behind it.
Again, for the purposes of smaller Lemmy instances (< thousand users), there is hardly a requirement for any kind of load balancing. Having one, is my definition and rationale for calling it fancy. It’s over-engineering a solution that is not a problem for majority of instances. Your arguments thus far would apply to larger Lemmy instances, even Reddit-lite, but that’s not what federation is about. It’s not a bunch of large instances forming a fenced community and gatekeeping the tiny ones out.