It feels like more Lemmy apps are going to make their way on to the app stores. With more apps, comes more people. More people, more API calls. How do we scale this server and hopefully all of the others to come, financially?
There are some REALLY interesting Podcast 2.0 features in the works. Especially using “value4value” and “boosting” as a way for listeners to tip their favorite podcasts and fund them directly. I wonder if somehow we can learn from it?
For those who do not know, hopefully these Podcasting 2.0 features will help podcasters continue to thrive in world where companies like Spotify and Amazon have decided to destroy our incredible open and free podcast networks by making “exclusives” and putting them behind paywalls that don’t follow the open standards.
I’d really love to integrate Podcasting 2.0 RSS and the fediverse. How cool would it be if every podcast episode just had its own place in the fediverse with a place to chat and it all worked together somehow automatically.
I dunno. Just a thought.
Here’s some info:
I just look to the microblogging side of the network (which has about 10 million total users) as a case study.
The ideal situation? More nodes are added to the network to spread the load and control away from a few very large and very expensive instances. The realistic situation? Some instances manage to secure external funding (such as mastodon.social) and grow extremely large at the expense of smaller instances that shut down from a lack of users and funding. Decentralized protocols like the fediverse and email are not immune to centralization thanks to lazy users who join the biggest instance. My pessimistic outlook is that the Fediverse will eventually become like email, with a few very big instances and a lot of spam making it difficult for smaller instances to enter the network. Enjoy the fresh new internet feeling while it lasts and move on when the platform starts to decay.
This is the realistic take I’ve been at for the two weeks since I learned all this existed.
Mastodon users are shocked and dismayed at the news of Meta using ActivityPub for their Twitterclone. And while it is sad, it’s the inevitable outcome hurtling forward.
While I loathe to admit it, this is just how communities seem to behave. Just like bacterial or fungal colonies on agar, the centers die from waste and lack of resources while the edges expand, and unless some larger force displaces some of the members to another plate, the culture will expand until it dies. This is why many of us moved from Reddit and other social media sites, we sort-of sporulated and rode the air currents to another petri dish.
The reason this system is unique, is that unless someone successfully patents and demands money for the software itself (which would be a legal nightmare to do at this point, if I understand correctly), we can rinse and repeat this process if and whenever it becomes necessary. Should lemmy.world become too congested, underfunded, or take the path of commercial giants like Facebook, Reddit, and Twitter–anyone can run up another server for their small group and start cultivating a new community, or move to any of the other growing communities.
There will be content and connections lost along the way, sure, but that’s just part of the impermanence of life, which in my opinion is part of the fun!
Edit, additionally: Believe it or not, some communities are self-limiting and find a harmonious way of existing within the ecosystem–like instances that focus on special-interest discussion groups that share a common theme.
I’ve been happy with lemmy.world and Ruud’s management, so I started contributing $2 a month to his patreon. I never paid a cent to reddit, but I want this place to succeed.
I think a Wikipedia donation system could work. Just takes a percentage of the core users to contribute.
Edit: https://opencollective.com/mastodonworld or https://patreon.com/mastodonworld
Which one is better to donate through? Patreon or OpenCollective?
Looks like Patreon takes a 5% cut while OpenCollective doesn’t appear to take a cut of the money.
Well, lemmy isn’t reddit, if one instance is down/closed then there’s a thousand other ones where you can go. So there’s no one big server that be overloaded from api calls - more like a million of them sharing the load.
As far as funding goes, each instance would decide on there own, but in the end most of them would settle for a patreon page or something similar.
deleted by creator
I think an API call to a server is less demanding than visiting or scraping the site. So I don’t think a 3rd party app is going to cause more issues than the traffic itself, which the hosters already have figured out. Reddit issues with API calls aren’t that they cause increased server load, it was that they didn’t get to serve you ads or collect your data. Lemmy doesn’t do either of those so that isn’t an issue.
deleted by creator
As of now, it’s all up to those who volunteered to host these open/large instances. They can accept donations but if they decide to shut down an instance, it’s gone. In the future I feel being able to gracefully handle instances disappearing would be the best bet. Financial reasons aren’t the only reason this could happen. Too many users could in theory break instances, as you can only scale vertically so much and at the moment I haven’t seen any talk of successful horizontal scaling. If users of an abandoned/deleted instance could easily move to another with minimal data loss this would mitigate this issue.
For long term viability, my opinion is legal entities (corporations, non-profits, etc) should be setup to handle larger instances that arise. They’ll operate as non-profits do, taking donations and hiring people to do the work that needs to be done. Expecting lone sysadmins to handle large user bases without some legal status/protection is a recipe for disaster. This also gives these larger instances a better standing to work within the current systems that will start asking questions/regulating if things get too big.
As for bringing in new users, these apps will have to make the process easier. It’s up to these apps to educate people or link to materials to educate people on the fediverse. These apps should be made to try and move users to instances that have the capacity to handle it and offer options. Yes, some users might find the fediverse and instances overwhelming but this is a common story with new things. Expecting everything to conform to how users currently operate is more for business interests, where user growth is a requirement for increased earnings and friction is bad.
The main concern with centralizing is once you have lemmy instances become centralized you arrive at the same position as reddit. What’s to say the largest lemmy instances won’t hold their instance hostage? Sell it to a corporation who liquidates it for the data and sees running the instance as a loss? Start defederating or limiting federation with other instances with malicious motives? If there is dozens of these larger instances, this will be easier to mitigate than the current handful of instances. It’s best if things are more decentralized, this is the goal of the fediverse after all.
Overall, lemmy isn’t ready for mass adoption as it stands. More work will need to be done and yes this is in “the future”. The current user base spiking doesn’t change the fact the code isn’t there, nor the fact code takes time. At the moment, you either suffocate instances by becoming too popular (lemmy.ml) or adapt and help contribute to get to the point where a large user base can be handled (such as mastodon has done).
The main concern with centralizing is once you have lemmy instances become centralized you arrive at the same position as reddit. What’s to say the largest lemmy instances won’t hold their instance hostage? Sell it to a corporation who liquidates it for the data and sees running the instance as a loss? Start defederating or limiting federation with other instances with malicious motives? If there is dozens of these larger instances, this will be easier to mitigate than the current handful of instances. It’s best if things are more decentralized, this is the goal of the fediverse after all.
This is definitely a valid concern but it’s not a problem right now! With the user numbers currently, it’s not monetizable. There are a few large instances poised to take up the load so even if one instance sold out, the rest would carry on!
I was chatting with someone about this earlier today. It would be great if there was a fan-out system something like icecast crossed with bittorrent, so people could contribute VPS or home internet to propagate Lemmy traffic. That would require some crypto signatures in the protocol to make sure the messages weren’t tampered with, of course.
It would be nice if developers would stand up their own instances for the apps to default to. They will be in the best position to collect funds directly from users in a way they are used to.
Maybe a little off topic but the briar project recently released their briar mailbox feature that allows what looks like something of a briar server that is deployed via android app. You connect to it via qr code and keep it always powered and connected to the internet. With so many people having android phones that they aren’t using anymore I think it would be awesome to continue this and allow people to deploy their own instances via second android device + android server app.
Developer here. I’ve been looking at the API calls made by the app, and I’ll try to give a good example of what is going on:
To be honest, you’re probably not going to see a drastic change in API calls right now. The only things that you are calling the API for are:
- Load items in the feed
- Load post/comments
- Load profiles
- Submit votes
- Submit comments
- Submit posts
- One initial call at app launch to obtain user info (subscriptions, settings, saved posts, etc, lemmy’s API gives you all of this in one call)
This is about the same use that you’re going to see in the actual web version.
While there may be upsides and downsides to how they are doing it right now, you can get pretty much all of the info you need through one API call. For example, if I get a post, the response will include most of the user info, most of the community info, and obviously all of the post info, plus more. I don’t need to make separate calls to retrieve all of that data.
Same goes for user info. In just one call, I can retrieve all of the information as far as subscriptions, moderated communities, user settings, and more without having to make a separate call for each one.
The issue is going to be mainly just the influx of traffic in general, not the apps themselves from what I can tell.
I’m also including the app’s name in the user agent so that if something were to ever become a problem, anyone can reach out and discuss what they are seeing so that it can be corrected.
Edit: I think one of the biggest issues that larger instances may run into is data usage. Nearly all photos that get posted to Lemmy are hosted on particular instances instead of services like Imgur. There’s ups and down to both, but in the short term I can see this possibly as being an issue if instances are not being ran on providers who are lenient or provide their data at low costs.
I think donations are sufficient.
Lemmy doesn’t seem to be too hard to run. Current popular instances run on HW that costs well under 100EUR monthly which well within reason for crowd-funding.
I see you’re getting downvoted, and I do have to agree that it’s a pretty optimistic take. With traffic even a tenth what reddit gets, the costs would be significant.
Now it’s true that eg. Wikipedia can handle massive server load on a donation model, but I think the utility from Wikipedia is more obvious and more amenable to attracting donations. I think it’s a good idea to think about palatable monetization options early on, so we can avoid ending up in a situation where the experience has to suddenly get degraded by intrusive ads or whatever.
If costs sky up for an instance, they can shut off user inscriptions and users will be forced to join another instance, like lemmy.ml did soon after the reddit exode (even though it was server limitation rather than pure costs, but the same logic applies).
So sure, some instance will grow bigger than others, but an instance should never reach a point of non affordable costs. Users will have to accept that some X funding is required to make the user base growth possible.
And for server shrinking to be possible, Lemmy must allow some sort of account migration. If for any reason half the funding of a server goes away, the admin must be able to rate limit their server until some users migrate away (any sort of rate limitation will be enough for some people to look elsewhere). But if migration is not possible then a server might get locked in a state of heavy costs and nobody being able to leave without creating a new account.
It feels like more Lemmy apps are going to make their way on to the app stores. With more apps, comes more people. More people, more API calls. How do we scale this server and hopefully all of the others to come, financially?
Look to Mastodon for what will happen. It went through a large migration from Twitter in the middle of November '22 (me included). The few established instances got slammed while new ones started. People started to move from larger to smaller instances that fit there needs. Many instances started accepting donations to cover costs and are in a good place. The same will eventually happen to Lemmy.
The BIG difference is the API cutoff on July 1st will cause many to migrate the first week of July. There will be some pain and some inexperienced admins may just give up.
I’d really love to integrate Podcasting 2.0 RSS and the fediverse. How cool would it be if every podcast episode just had its own place in the fediverse with a place to chat and it all worked together somehow automatically.
Someone may make it once there is an actual spec.
What you’re mentioning is key. Lemmy is exploding right now. Imagine when all third-party apps cease working… It’s gonna be crazy.
Over 90k new users in the past 36 hours.
And oh boy does it show. There’s several lemmy instances struggling to keep up with federation right now.
The BIG difference is the API cutoff on July 1st will cause many to migrate the first week of July. There will be some pain and some inexperienced admins may just give up.
There’s already a bit of that, considering that some of the instances effectively imploded under the few users that did move over.
