Raphael Lullis

ADAPT: the server to bring back the (Social) Web

Raphael Lullis — Tue, 03 Mar 2026 01:40:39 +0300

The blog has been quiet - partly because my usual beat has lost a bit of space to things that I can not Actually Indicate, and partly because frankly nothing of major interest was going on in the Fediverse. Zuckerberg plans to become an infrastructure company fizzled out due to (or thanks to?) European regulations. Reddit continues its slow descent into an exclusive forum for chatbots talking with each other and finding hooks to plug some advertisement. Bluesky didn’t keep the post-election hockey stick growth, but managed to hold on enough of the userbase and hasn’t completely flamed out.

Even Communick had reached this weird stasis: enough customers to pay the server bills but not enough growth to justify more work on it. After going through one Social Dilemma, two Twitter Diasporas and countless Mastodon and Lemmy instances getting abandoned for insufficient support, maybe it was time to accept that Federated Fetch was not going to happen.

But by the end of last year, something started to change in a way that people could no longer ignore. What was previously seen as Traditional Corporate Greed has become a true concern for a full-on Surveillance State. With Governments in the US and Europe pushing hard for age verification requirements, content moderation mandates, and platform-level accountability laws; centralized platforms are becoming targets themselves. When you concentrate hundreds of millions of users into a single system under a single jurisdiction, you make life easy for the Powers That Be to control you.

And it’s not like there are any innocent actors in this game, either. That’s the deal platforms accepted when they decided that growth was the only metric that mattered. And the companies that gave away control of the communication channel for convenience and optimization are finding out what it means to have no leverage: they’ve watched their reach get algorithmically throttled, their content policies changed without notice, and now they’re watching the platforms themselves become political liabilities.

What then?

Of course, this pressure has caused some movement in the direction of open and federated systems. Communick once again started getting a more constant influx of signups. But no matter how far I extrapolate those growth curves, it’s clear that the vast majority of people are not going to take the initiative to move anywhere. If we want the Fediverse to be universal and a credible threat to the Big Tech platforms, we need to make it interesting for the “cottage industry” that grows on every platform: small businesses looking for a way to reach their customers, content creators who live and die by the size of the audience, other independent developers looking to build a product that serves a niche, etc. These are the people who suffers the most with enshittification and artificial limitations of the platforms, and these will be the first to try any alternative that seems better than the status quo.

These problems will not go away if they leave one corporate-controlled platform and move to another corporate-controlled platform. Leaving Twitter for Bluesky is not a solution. Leaving Facebook for TikTok even less. The only logical response is to stop depending on infrastructure they don’t control. What we need is to bring back the web, a loosely interconnected collection of servers that can withstand individual failures.

Time to ADAPT

One of my many side projects is a website called CareerCupid. It is a website that could be described as “OkCupid for Jobs”. People sign up, answer questions about their values, work styles, needs and wants in a company. The answers of all members are compared against one another and then we give a “matching score” saying how well you would work together.

To bootstrap the network, I wanted it to make it easy to participate even if they never saw the website. So I had the idea of integrating a bot which people could be followed on the Fediverse, and I wanted to give one “actor” for each user that was on the website.

Could I have done that by setting up a Mastodon instance and setting up bot accounts? Yes. Was that what I did? Of course not. CareerCupid is a Django site, so I’ve built my own library that could integrate with it directly. In the process of getting that “cupid bot” interacting with the whole Fediverse, I ended up learning and building enough of ActivityPub and the related set of standards: RDF/Linked Data, Webfinger, Authentication via HTTP message signatures… but above all it helped me understand that all the building blocks that make the “Social Web” are also the building blocks of the Semantic Web. The idea of “different platforms that can talk with one another” is already powerful, but the real insight is that this is mostly a side-effect of RDF - and that integrations can be as trivial as defining a custom vocabulary and letting the world know about it. There’s a deeper implication here around Linked Data and what it means for AI applications - but that deserves its own post.

That’s basically how the Django ActivityPub Toolkit came to existence. It’s not meant to be yet-another service to emulate the proprietary networks (though it can be, as it provides a Lemmy-API adapter), but rather a library and application server that can be used to let you integrate your existing applications and let you build on the social graph (the same one Mastodon, Flipboard, and a growing number of other services already use) without requiring you to bootstrap a whole network every time.

Help me take this to the next level

I’ve been working on this alongside client work, which is the wrong way to build infrastructure when things are changing so fast. I want to make this my full-time job, ideally by finding a company that wants to hire me to keep working on this. If that’s not the right fit yet, I’m seeking 4 to 6 companies to each commit $2,500/month: 10 hours of dedicated development for your specific use case, office hours for your engineering team, and first right of refusal if the project gets acquired. Think of it as sharing the cost of a dedicated infrastructure engineer - at a fraction of what a full hire would run any single one of you.

I’m not looking for donations or one-time grants. This is closer to buying an option: a small bet on infrastructure that might matter a lot in two years, shaped in part by your own requirements, for less than you’d spend on a single engineer’s weekly salary.

The companies I have in mind are ones with an obvious strategic interest in federated infrastructure: telcos who were turned into dumb pipes by Big Tech, news publications who’d like to take back control of their distribution, browser vendors who’d rather not depend on platform APIs. If that’s you, get in touch.

Can we fix job sites?

Raphael Lullis — Tue, 11 Feb 2025 20:05:46 +0100

(I want to try something different here. Instead of a fully fleshed out post, I’d like to just start with a draft of some ideas and I hope that it is enough to generate a conversation. I’ll take the relevant responses and use them to keep improving this article)

StackOverflow is nowadays famous for being the site that you only go after you’ve given up ChatGPT to stop hallucinating, but one of its lesser known products was StackOverflow Careers (sorry, couldn’t find the link to the original announcement), which absolutely raised the bar in how job sites worked. It was one of the first sites every job listing had to provide information about the hiring company and a direct contact. This might seem weird for anyone born in this century… but boy, how refreshing it was in 2010 to be able to see a job listing who was not from a recruiting withholding information.

It seems like the social media and Big Tech broke job searching again. Companies that are hiring are drowned by “content creators” on LinkedIn or taking a “spray and pray” approach by posting to whatever site is popular at any given month. Job seekers are now playing a game of cat-and-mouse with AI tools, figuring out whether their changes are going to help them stand out or make their CVs look exactly like everyone else’s.

Here is my tentative list of things that should be created for a job site that fixes those issues:

Remove siloes and walled gardens.

This one is obvious. People shouldn’t have to sign up to any specific site to access information. Applying could be as simple as sending a message via email. With protocols like ActivityPub, we could have a simple “like this job posting indicates interest”.

I am not saying that it has to be built on ActivityPub, but anything that brings some form decentralization (maybe Solid?) would certainly be an improvement over the status quo.

Create a standard measurement to allow people to know where they are, skills-wise.

This could be one good use of gamification. Imagine that every job listing had a well defined matrix of required skills to do that job and at what level of expertise. People could figure out even before applying whether they had a chance at the job or - most importantly - they could see exactly what is separating them for the next step up in the professional career.

If this is properly implemented, we would be able to get rid of the whole “CV analysis” and all forms of subjetive evaluations.

Make it functional for people who are not looking for a job

If you are happy at your job and you prefer to do your networking offline, there is a good chance you only keep your LinkedIn account for the entertainment value. For most, joining and participating in a professional-focused site makes sense only when job seeking. Sure, that by itself is already enough to bring a massive user base, but couldn’t we make things even better by also making things interesting and useful for those who want to help others?

Just an example: there are so many companies that create incentives for employees who make good referrals. Couldn’t we perhaps think of a system that enables established professionals to do that at a wider scale?

Or consider the ATS systems that are (mostly) managed by the recruiters. They are built in a way where recruiters are kept in control of the top of the funnel, and prospects are only introduced to the hiring manager after they have passed some pre-screening interview. This is not only slow, but it is also inneficient: even the best of recruiters can not evaluate a potential candidate better than their eventual team mates and direct reports. What if we could simply get rid of pre-screening interviews by allowing everyone in the team to join in the triage?

Reduce the stakes: prospecting should not indicate commitment

The days of being rewarded by being loyal to any particular company are long over. This means that the default state for anyone should be “do your best, prepare for the worst”. Keeping a constant pulse on the job market and being open to talk about new opportunities, even when you are perfectly happy with your employer, should be normalized and not automatically treated as a sign that you are “ready to jump ship”.

Default to transparency

There is so much information that a company can disclose that could and should be available before we even get to the “apply now” button: who is on the team? What is the current org-chart and who are they are hiring for? How are they going to assess candidates for the skills being required? Can this evaluation be done prior to the application?

No gated features to extract revenue from job seekers

Paying for reach or access to a valuable contact is criminal.

More?

Thoughts? Suggestions? Let me know on @raphael@communick.com

When information is cheap, attention is expensive

Raphael Lullis — Thu, 26 Sep 2024 23:41:39 +0200

TL;DR: I have a number of Lemmy instances meant for discussion groups around specific topics. They are not being as used as I expected/hoped. I would like to set them up in a way that they can be owned by a consortium of different admins so that they are collectively owned. My only requirement: these instances should remain closed for registrations and used only to create communities.

During the height of the “Reddit API protests”, I had come up with The Grand Plan to bring a sizeable part of Reddit’s userbase to the Fediverse alternatives. The Grand Plan was as follows:

Identify the largest segments of community interests that are popular on Reddit.
Create different Lemmy instances for each of these identified topics.
Build a tool to let users sign up to Lemmy and automatically subscribe them to the relevant communities based on their favorite subreddits
???
Profit!

Much to my own frustration, The Grand Plan has not progressed much beyond step #3. Reasons for this failure abound, but there are two flaws that are exclusively my fault:

Because I wanted to keep the instances focused around topics, registrations were closed and I was the only one who could create communities on them.
Because I was promoting these instances along with Communick, I led people to believe that these instances were going to be part of some commercial scheme.

I am now looking to find a way to fix both of these problems. I would like to find a way to have these instances under the governance of some collective or consortium, and to make them part of an unified response for anyone that is looking for a Reddit alternative.

These are the instances that are active and running:

https://soccer.forum (Football)
https://nba.space (NBA Basketball)
https://nfl.community (American Football)
https://matchpoint.zone (Tennis)
https://hardware.watch (Consumer Electronics)
https://poweruser.forum (Software Support)
https://selfhosted.forum (selfhosting)
https://blockchained.world (crypto)
https://netheads.online (VoIP)
https://sfw.community (Equivalent to Reddit’s SFW Network)
https://level-up.zone (Gaming)
https://metacritics.zone (TV & Movies)
https://healthy.community (Health & Fitness)
https://indiehackers.space (Business and Entrepreneurship)
https://viewfinder.pro (Photography)
https://hi-fi.community (Hi-Fi Audio)
https://style.land (Fashion and style)
https://gearhead.town (Car and Motor)

My only “requirement” to make this happen is to keep these instances exclusively as a home for groups, so registration should be closed at all times. Unlike with Mastodon, I believe that keeping groups/community actors connected to instances of related interests for the following reasons:

Make it easy for content discovery. Things currently are very ad-hoc and people tend to create communities for their interests if they do not immediately find them on their instance. If we have a better separation between “instances for users” and “instances for groups”, onboarding would be easier and unnecessary duplication can be avoided.
Mitigate potential issues from political fighting between users from different instances. If instance A has a conflict from instance B and wants to defederate and if a community is popular on B, then the “natural reaction” would be for users on A having to recreate the communities. This is not just a problem for users of A and B, but everyone else on other instances. If the communities are on a neutral ground, they would be less likely to become collateral damage.
If we have instances that are more focused on hosting users, it would be easier to distribute incoming users around.

To repeat: I am not doing this because of financial issues (though I wouldn’t mind if you want to become a sponsor) or because I am unable to keep managing them. The only reason that I am considering this is to see if I can get more alignment with other instance admins and organizations that might be interested in collaborating.

If you are part of an organization that could be interested in joining, or if you are an individual who would like to help, please feel free to reach me on Mastodon.

A Plan for Social Media - What? ✅ Why? ✅ Who? 🤷

Raphael Lullis — Mon, 20 May 2024 23:45:00 +0200

This is the final part of the series calling for a different approach in the development of Social Media based on open standards. In the previous post, I argued that we could build applications that relied less on functionality provided by the server and put the client in control. The server software needed would be responsible mostly for authenticating users (i.e, keeping track of user identities) and sending and routing ActivityPub messages.

I want to close out this series with a list of market players who could be interested in trying out this strategy. Unfortunately, investment in new technologies is more dependent on hype than on a clear value-based evaluation, which means that someone like Mozilla backing down of their plans can be seen that the momentum for the open social web is reduced. Couple that with the fact that “Fediverse Culture” is still largely anti-corporate and averse to “profit-motive” ventures and it’s hard to imagine a large number of companies eager to place some bets into this game.

Still, I do believe that there is an opportunity here. Just look at how much Facebook is investing strongly on Threads and its (so far, credible) commitment to interoperate with other ActivityPub-based systems. Threads has been promoted as the “Twitter killer”, but to me, this is more than just a social network with billions of users trying to snag a hundred million more from a rival; Zuckerberg’s investment in Threads should be seen as a medium-long term play to move away from being a company that runs a panopticon to an infrastructure provider for other companies. This move makes sense to them. We are already seeing how AI-generated content is ruining search and poisoning the well of community recommendations. It seems reasonable to believe that social media will be moving away from the “global town square” and give room to direct conversations between people in small groups that already have a established connection. With that in mind, who should we be watching for as potential businesses coming up with new initiatives in the space?

Browser vendors

Mozilla’s initial effort failed. And, unlike Google, they do not have a history of persuing a space multiple times. They have taken many shots at different ideas, but I do not recall any project that has been repeatedly tried.

What about the other browser vendors, though? Apple still leans hard on the idea that their platform is more secure and protective of users’ privacy. They are also known for “not doing social media”. But the appealing aspect of a protocol-based approach to it is that they wouldn’t have to build the network, they would just need to ensure that the experience is superior. Consider how the whole “Reddit API protests” started due the most popular client (Apollo) being so much better than the Reddit’s own client that it was hurting Reddit’s revenues. Would it be too crazy to have Apple re-hiring Christian Selig to make “Social Web Client” built into iOS or Safari?

If not Apple, maybe other Chrome-based, privacy-focused browsers? Brave could use some investment in social as a way to reinvegorate its crowdfunding platform and could also leverage its in-device ad inventory, turning into a system that let advertisers find their audience and connecting them directly through social media. Users of DDG and Vivaldi with a built-in social browser would certainly have increased usage and contribute to their revenue streams.

Website builders

One of the worst aspects of the “walled gardens” was that it made impossible for independent professionals and small businesses to reach their customers unless they paid to promote their own content.

Whether you are the owner a small restaurant, a farmer selling produce at a local market or a wedding photographer, having all these separate social media networks means that you have to keep an online presence on each of them.

Being able to have social web based on open standards would mean that people would be able to take back control of their online presence. As long as you have your own domain, you are in control of how you interact with anyone else.

If Facebook’s plan really is to pivot to an infrastructure provider, it would be wise for companies like squarespace, Wix and Weebly to preempt their move and create offerings where their customers can simply provide their domain and get an account on ActivityPub-capable server. This move is already happening in the open source world: Wordpress has already created an integration with ActivityPub that lets people follow and interact with a blog from Mastodon and other Fediverse software, and Ghost also started working on it, presumably with the intent of adding monetization options for content creators that want to have an alternative to the likes of Medium and Substack.

Domain registrars

Identity and reputation is an integral part of decentralized systems. If you receive a message from a recruiter that claims to work for ACME, inc., but the email message is sent from “jane.from.acme@hotmail.com” you will be a lot less likely to trust it. With current social media, we are still in this weird situation where brands and famous individuals risk their names taken by squatters and have to rely either on “blue checks” or having enough clout to be able to claim their name back, which can be a long process.

The majority of domain registrars offer auxiliary services for everyone that is buying their own domain. The current suite of “Fediverse” software is still too big for these companies to be able to offer, but having a simpler, client-first system for ActivityPub would mean that they could partner with other service providers or even run their own, like they do with email.

Final thoughts

I don’t really have a definitive conclusion here. I just wanted to lay out some thoughts based on my involvement with the Fediverse and from observing the direction that its development has taken. While it’s impossible to prove it, I sincerely believe that we have enough evidence to conclude that the current server-centric approach is not going to be able to take us to the next level in mass adoption. To support this belief, I can say that Communick is no longer going to offer Managed Mastodon hosting (for new customers, existing ones do not need to worry) and will instead focus on promoting Takahe, because it at least provides a solution where one server can be used to as a host for multiple domains. I also starting exploring some ideas on a fork of Takahe to make it a testbed for ActivityPub C2S.

If you are interested in the future of social media and would like to support my work, here is how you can help, in order of increasing commitment:

Join Communick. For $29/year, you get access to a suite of services that includes an account on Mastodon, Matrix, Lemmy and also listen to music on Funkwhale.
Become a sponsor.
Hire me. I’ve been working with web and large-scale distributed systems for almost 20 years, but most important I have a good track record for delivering customer-first, product-focused work. Whether for full-time or part-time positions, I am available for any short-term or long-term project.

A Plan for Social Media - Rethinking Federation

Raphael Lullis — Sat, 18 May 2024 15:45:00 +0200

This is part of a series of posts about the current state of Social Media. I talked about Mozilla’s failed strategy to get people out of the corporate-controlled networks and into the open alternatives and made a quick demonstration of how ActivityPub is used to power the “Social Web”. I wanted to talk about these things before I get to the main point of my thesis: the “Fediverse” as a concept of “servers who can interact with one another” is antithetical to the open, social web. We need to go back to the drawing board and re-design our software in a way that creates bottom-up connections, instead of the current top-down approach.

I’ve already talked about this before, but to reiterate: aside from cases where people are self-selecting or acting as a representative of a larger group - i.e, they are part of that network because of their affiliation - the federation model has little to do with how people identify themselves and their relationships with others. It’s no wonder so many people have difficulty “choosing an instance”: they are implicitly being asked to pledge alliance to a group they never met before, and are told (literally!) that this will be a crucial part of their identity in this social space.

Other protocols like Bluesky have solved this issue, and there has been work on ActivityPub servers that decouple their domain from the user’s Actor ID, but the majority of servers still operate on the idea that users that want to use their service must “belong” to them.

Servers are expensive to operate

Centralized services benefit from a client-server architecture because of economies of scale. The marginal cost per extra user is neglibile compared to the initial cost of setting up a server, so when your service is being used by thousands of people, it is more than reasonable to focus on having a system that can grow by “throwing more machines at it”.

This is not necessarily true for distributed services. The obvious example is the “single-user” instance: any service provider will have to provision a whole Mastodon server, with its own database and its own processing queues, which can not be shared with other people.

While it’s undeniable that the developers of the leading projects have largely delivered on the promise of “getting rid of walled gardens”, it’s also true that our user experience is still determined by the type of server that we are using. This happens because projects have started by emulating one of the closed social platforms and “porting” it to ActivityPub.

Mastodon (Pleroma, Misskey): like Twitter, but can federate!
PixelFed: Federated Instagram!
Lemmy: Reddit, but federated!

While there are some projects that try to unify some of these aspects (kbin, for example, provides both microblogging and group discussion), people still need to keep multiple accounts in different servers if they want to do different things. You can not follow someone’s Pixelfed stories from your Mastodon account, even though a less “server-centric” ActivityPub application would make this perfectly possible.

The server is the wrong place for application logic

Some issues from Mastodon:

When you first sign up to a account, you might be greeted with a list of recommended users to follow. This list can be configured by the server admin, but in any case it will only be able to show you accounts that are from the same server as yours or from a server that already “federates” with it.
If you visit a profile from an account from a different server, you can only see their posts that have directly interacted with your server. If you are on a small server, chances are that you will see only the profile information, but none of their previous posts, and you will have to browse to the “remote” instance to see their actual timeline.
When you run a search query, you are running the query through the server, and again the results are only a reflection of the data that has been “seen” by the server.
The “trending hashtags” feature only works for tags that have been explicitly approved by the instance admin.

Lemmy:

The server provides different sorting methods for posts and comments, such as “Hot”, “Active”, “Newest”, etc. There is no way for the user to define their own sorting method.
There is no reasonable way to define policies to handle votes, aside from instance admin being able to enable/disable “downvotes”.

All these issues could’ve been avoided if the decision-making process was delegated to the client. It’s nice that the server can make search queries, but it would be even better if the client could choose a separate, “global” search engine. It’s nice that the server can make recommendations for accounts to follow, but it would be even better if I had a menu pointing me to external services that are dedicated to this task. By forcing the user to do everything through the server, we become needlessly dependent on it.

Networks are smarter at the edges.

The “server-centric” approach to the application development is at odds with the nature of distributed systems. We need to stop treating the client as a dumb terminal and give it as much power (and responsibility) as possible.

The curious thing is that ActivityPub has already quite a bit a work of done to allow applications to do exactly that. The W3C specification has a “Client-to-Server” (C2S) API to standardize how clients can interact with their inbox and outbox, if we flip the architecture of the applications and make the clients responsible for the “smart” parts of the system (content discovery, timeline management, filtering rules for moderation and protection, etc) and let the server squarely with the single responsibility of message routing, not only we would make of the application development faster, we would quite possibly solve the scalibility and TCO issues of the server. If we consider that XMPP servers can handle millions of concurrent users on a single node, it’s easy to realize that this architecture would make large scale deployments of Social apps cheaper.

There is also another benefit for server operators: by turning the server into a “simple” message relay, there would be no data stored in the server and these messages could even be end-to-end encrypted, which would mean that operators would have less liability risk.

Lastly, we can not ignore one potential improvement in regards to privacy. Less data on the servers means there is less data available for the likes of Google, Facebook and OpenAI.

If we have the client as the center piece of our social web, why not build it already in the foundation of the web client that is already used by virtually everyone?

Going back to our very first post in this series: what did I expect to see from an organization known for developing a web browser, if not an initiative to have these features in the browser? Instead of experimenting with their own instance, Mozilla could have “executed with greater agility” if they worked on a browser extension that started a simple “Actor ID” follower. That alone would already be as useful as a “RSS reader for the social web”. After they got this in place, they could start refining it. Add login capabilities, notify user of received messages, post messages to public timeline, etc.

And for bonus points: after we got these building blocks in place, there would be nothing stopping us to extend the functionality to other networks. What better way to prevent data tracking from the likes of Reddit/Facebook/Twitter than by simply having an alternate client inside the browser?

There are extensions like libredirect that take links to “problematic” sites and point to alternative frontends. If users could have such a service built into the browser, we would be effectively denying the companies any chance of tracking users extensively.

A Plan for Social Media - Understanding ActivityPub

Raphael Lullis — Sat, 18 May 2024 03:00:00 +0200

This is part of a series of posts about the current state of Social Media and the Fediverse. In the first post, I described some of the issues with the shot that Mozilla took with their mozilla.social initiative. By the end of this series, I plan to have laid out a strategy that can be more successful. But before doing all that, it will be good to have a very fundamental understanding of ActivityPub, the protocol that enables interoperability between all the different services. Let’s get to it.

ActivityPub, actors and message boxes

If you want a proper introduction to the protocol, a good resource to start is activitypub.rocks. There we can find this:

The diagram introduces us to the three main concepts in ActivityPub:

Actors as the entity being addressed by messages
The inbox is the URL where the actor can be reached
The outbox is the URL used by the actor to post new messages

This looks simple, but how do we get from these simple concepts into an useable application? Let’s say that we want to find out what was the last message that I posted on my @raphael@communick.com account.

The first thing we will figure out is that my handle is @communick.com, but the server is mastodon.communick.com. How do we get from one to the other? The answer is Webfinger, a protocol to help us with service discovery via the web.

Let’s query communick.com and find out where to reach my account:

$ curl "https://communick.com/.well-known/webfinger?resource=acct:raphael@communick.com" -I

I am cheating a bit here. I’m using the -I flag is to show us the response headers, because I know already that’s the relevant part of the response:

HTTP/2 302
content-type: text/html; charset=utf-8
cross-origin-opener-policy: same-origin
date: Fri, 17 May 2024 01:43:01 GMT
location: https://mastodon.communick.com/.well-known/webfinger?resource=acct%3Araphael%40communick.com
referrer-policy: same-origin
server: uvicorn
x-content-type-options: nosniff
x-frame-options: DENY
content-length: 0

Ha! communick.com is telling us to go to mastodon.communick.com, and make the same query there. Let’s try it, this time we expect the actual data:

$ curl "https://mastodon.communick.com/.well-known/webfinger?resource=acct:raphael@communick.com" | jq .

And the response is:

{
  "subject": "acct:raphael@communick.com",
  "aliases": [
    "https://mastodon.communick.com/@raphael",
    "https://mastodon.communick.com/users/raphael"
  ],
  "links": [
    {
      "rel": "http://webfinger.net/rel/profile-page",
      "type": "text/html",
      "href": "https://mastodon.communick.com/@raphael"
    },
    {
      "rel": "self",
      "type": "application/activity+json",
      "href": "https://mastodon.communick.com/users/raphael"
    },
    {
      "rel": "http://ostatus.org/schema/1.0/subscribe",
      "template": "https://mastodon.communick.com/authorize_interaction?uri={uri}"
    },
  ]
}

So, now we know how my account actually lives. The response is describing different links that would be relevant to someone who want to interact with this account. From those, the most interesting one is rel=self`, which tells us which url is used to represent my profile as an actor. It is also indicating what mime-type should be used to make the request. Let’s try it:

$ curl https://mastodon.communick.com/users/raphael -H 'accept: application/activity+json' | jq .

And the response - with most attributes ommited for brevity:

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://w3id.org/security/v1",
    ],
  "id": "https://mastodon.communick.com/users/raphael",
  "type": "Person",
  "following": "https://mastodon.communick.com/users/raphael/following",
  "followers": "https://mastodon.communick.com/users/raphael/followers",
  "inbox": "https://mastodon.communick.com/users/raphael/inbox",
  "outbox": "https://mastodon.communick.com/users/raphael/outbox",
  "featured": "https://mastodon.communick.com/users/raphael/collections/featured",
  "featuredTags": "https://mastodon.communick.com/users/raphael/collections/tags",
  "preferredUsername": "raphael",
  "name": "Raphael Lullis",
  "devices": "https://mastodon.communick.com/users/raphael/collections/devices",
  "publicKey": {
    "id": "https://mastodon.communick.com/users/raphael#main-key",
    "owner": "https://mastodon.communick.com/users/raphael",
    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA2Qsjv7JwN0vnEk6Vtqiv\n/S6qk7yGmynpGZW1F/qiuzpnoLfF18czx0UFHU4imU8xv8eYDhvRUSj1nVwRmT/S\noIA0c/zlesBxkxMxtiml/dbaTUXxfgsa5SKtw6l8CaFv1IfpXAmc9T24RFiUVFGU\nWUDtKqWwYwT+QZiQG7YAF89ehEcXvbGajnQOnORgT3O+M4VOxMgNMzZsvtCTa9kg\nY1Ll56lk5XkOy/VauC5YmawAUDcfinpZVHjkKmVE+qaOg0ELduL6Vw42CADCN3TV\nHz0Ap6wJPeTj0ma+a2l8eE9MyqX7M7Y115kV2eyB0/BKMKdhMyLuNaocCFNjhm4y\ncQIDAQAB\n-----END PUBLIC KEY-----\n"
  },

}

Alright, now we are getting closer. Not only this response already provided the cryptographic public key used by my account (an important aspect for those that want to make sure that the messages I’m sending are actually from me), we have also have the urls that tells us what are the inbox and the outbox.

Let’s find out what type of nonsense I’ve been posting?

$ curl https://mastodon.communick.com/users/raphael/outbox -H 'accept: application/activity+json' | jq .

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "id": "https://mastodon.communick.com/users/raphael/outbox",
  "type": "OrderedCollection",
  "totalItems": 1474,
  "first": "https://mastodon.communick.com/users/raphael/outbox?page=true",
  "last": "https://mastodon.communick.com/users/raphael/outbox?min_id=0&page=true"
}

So, the people working on Mastodon are smart enough to know that they shouldn’t respond with all 1474 posts every time someone request the content of my outbox, so it tells us to look at the paginated results.

curl "https://mastodon.communick.com/users/raphael/outbox?page=true" -H 'Accept: application/ld+json' | jq .orderedItems.[].id

Again for brevity, I’m going to list only the ids of the posts.

"https://mastodon.communick.com/users/raphael/statuses/112453788068689232/activity"
"https://mastodon.communick.com/users/raphael/statuses/112451233768213990/activity"
"https://mastodon.communick.com/users/raphael/statuses/112450952761586193/activity"
"https://mastodon.communick.com/users/raphael/statuses/112450838604583727/activity"
"https://mastodon.communick.com/users/raphael/statuses/112450692646931646/activity"
"https://mastodon.communick.com/users/raphael/statuses/112450481357958805/activity"
"https://mastodon.communick.com/users/raphael/statuses/112450434346644443/activity"
"https://mastodon.communick.com/users/raphael/statuses/112450380678942390/activity"
"https://mastodon.communick.com/users/raphael/statuses/112450366779981942/activity"
"https://mastodon.communick.com/users/raphael/statuses/112450254434441343/activity"
"https://mastodon.communick.com/users/raphael/statuses/112445793173734858/activity"
"https://mastodon.communick.com/users/raphael/statuses/112445792468741280/activity"
"https://mastodon.communick.com/users/raphael/statuses/112445753422312223/activity"
"https://mastodon.communick.com/users/raphael/statuses/112445505614367554/activity"
"https://mastodon.communick.com/users/raphael/statuses/112427864602893406/activity"
"https://mastodon.communick.com/users/raphael/statuses/112412635359916476/activity"
"https://mastodon.communick.com/users/raphael/statuses/112411290416210974/activity"
"https://mastodon.communick.com/users/raphael/statuses/112410255917704070/activity"
"https://mastodon.communick.com/users/raphael/statuses/112409574645967938/activity"
"https://mastodon.communick.com/users/raphael/statuses/112409529723480739/activity"

And let’s take a look at the response for the first result (i.e, my most recent post):

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    {
      "ostatus": "http://ostatus.org#",
      "atomUri": "ostatus:atomUri",
      "inReplyToAtomUri": "ostatus:inReplyToAtomUri",
      "conversation": "ostatus:conversation",
      "sensitive": "as:sensitive",
      "toot": "http://joinmastodon.org/ns#",
      "votersCount": "toot:votersCount",
      "Hashtag": "as:Hashtag"
    }
  ],
  "id": "https://mastodon.communick.com/users/raphael/statuses/112453788068689232/activity",
  "type": "Create",
  "actor": "https://mastodon.communick.com/users/raphael",
  "published": "2024-05-17T01:17:11Z",
  "to": [
    "https://www.w3.org/ns/activitystreams#Public"
  ],
  "cc": [
    "https://mastodon.communick.com/users/raphael/followers"
  ],
  "object": {
    "id": "https://mastodon.communick.com/users/raphael/statuses/112453788068689232",
    "type": "Note",
    "summary": null,
    "inReplyTo": null,
    "published": "2024-05-17T01:17:11Z",
    "url": "https://mastodon.communick.com/@raphael/112453788068689232",
    "attributedTo": "https://mastodon.communick.com/users/raphael",
    "to": [
      "https://www.w3.org/ns/activitystreams#Public"
    ],
    "cc": [
      "https://mastodon.communick.com/users/raphael/followers"
    ],
    "sensitive": false,
    "atomUri": "https://mastodon.communick.com/users/raphael/statuses/112453788068689232",
    "inReplyToAtomUri": null,
    "conversation": "tag:communick.com,2024-05-17:objectId=1006185:objectType=Conversation",
    "content": "<p>I&#39;m working on a blog post where I argue that we should stop writing web applications that emulate &quot;traditional&quot; social networks and propose a different approach for <a href=\"https://mastodon.communick.com/tags/ActivityPub\" class=\"mention hashtag\" rel=\"tag\">#<span>ActivityPub</span></a> software, putting the client at the center of everything and making the servers just specialized relays of ActivityStream messages.</p>",
    "contentMap": {
      "en": "<p>I&#39;m working on a blog post where I argue that we should stop writing web applications that emulate &quot;traditional&quot; social networks and propose a different approach for <a href=\"https://mastodon.communick.com/tags/ActivityPub\" class=\"mention hashtag\" rel=\"tag\">#<span>ActivityPub</span></a> software, putting the client at the center of everything and making the servers just specialized relays of ActivityStream messages.</p>"
    },
    "attachment": [],
    "tag": [
      {
        "type": "Hashtag",
        "href": "https://mastodon.communick.com/tags/activitypub",
        "name": "#activitypub"
      }
    ],
    "replies": {
      "id": "https://mastodon.communick.com/users/raphael/statuses/112453788068689232/replies",
      "type": "Collection",
      "first": {
        "type": "CollectionPage",
        "next": "https://mastodon.communick.com/users/raphael/statuses/112453788068689232/replies?only_other_accounts=true&page=true",
        "partOf": "https://mastodon.communick.com/users/raphael/statuses/112453788068689232/replies",
        "items": []
      }
    }
  }
}

Et Voilà! We figured out how to go from my account handle, to the actual Actor URL, to the outbox, and finally to read a message that I have posted. An even more interesting exercise would be to make a similar journey from the other end and send a message to me, but to do that we need to get a bit deeper into how messages are structured, the cryptography involved and some other implementation details which are not at all related to the main point here. The important thing to take away here is the concept of actors and their inbox/outbox, which can be represented by URIs and discovered through established protocols.

Now that we know the basics, we are ready to take the next step. We will challenge one of the basic assumptions common to all current software using ActivityPub and flip the whole architecture on its head.

A Plan for Social Media - Intro

Raphael Lullis — Fri, 17 May 2024 20:00:00 +0200

About 18 months ago, Mozilla announced that they would explore a healthy social media alternative. Their started their efforts by setting up a server and putting up together a team to work on new applications for the ecossytem. Fast-forward to February of 2024, and they released a memo which said:

Our initial approach was based on a belief that Mozilla needed to quickly reach large scale in order to effectively shape the future of social media. It was a noble idea but one we struggled to execute. While we resourced mozilla.social heavily to pursue this ambitious idea, in retrospect a more modest approach would have enabled us to participate in the space with considerably greater agility.

The more cynical of you would be quick to argue that this memo is mostly corporate talk to justify their poor management decisions of the last years and to give them an excuse to jump into the AI bandwagon.

But then, there is this:

How healthy is a network that loses 40% of its active users in 10 months?

It seems that no matter how badly the tech companies screw up and no matter how many people show complete dissatisfaction with the status quo, the Fediverse as a whole keeps “struggling to execute”. Does this mean that Mozilla is right to just get out of the space altogether? Should we just give up on the idea of a healthier social media landscape? In this blog post, I’d like to make a deeper analysis of the core pieces of the Fediverse and:

show how Mozilla’s approach was flawed and unlikely to succeed even if they had more substantial growth
propose a different approach that “can be executed with greater agility”
make a pitch for potential businesses and players in the space who still see value and an opportunity to take back the social web.

Get comfy in the chair, because this is looking like a mini-series…

Mozilla’s mistake and the Pachyderm in the Room

Despite Firefox’s constant slippage in market share, Mozilla still has considerable influence as one of the stewards of web technologies and is widely regarded as the only large institution in the standard bodies acting with the best interests of people in mind - in contrast with the likes of Google and Microsoft, who are seen as pushing for the development of technologies that mostly serve their own agenda. So, when they announced their plans to explore the Fediverse, one could reasonably expect them to take a look at ActivityPub (after all, it is already a W3C standard) and join in the efforts to make technologies that leverage the protocol directly. It would have been completely aligned with their mission if they had, e.g, decided to use Pocket as a laboratory for “social web” features based on ActivityPub.

Instead, they announced they would be launching their own Mastodon instance. While at first this seemed like good news (Yay, look at who is joining us to the party!), this gradually has shown to be completely backwards in regards to a reasonable strategy. Let us count the ways:

Mastodon is too opinionated and makes for a bad experimental system

This is not something to blame the Mastodon developers for; what they managed to create with such a small team is actually impressive. But if Mozilla’s goal was to “explore a healthy social media alternative”, it would make a more sense if they started with something that could let them be more flexible.

For example, Mastodon (as most of the current Fediverse software) assumes that one server will only work for one domain, and that everyone’s identity will be tied to this domain. There is no system nowadays with a good solution to identity portability is non-existent. The best that Mastodon can do is simply to let people take their data from one server to the other, but the identity itself is not in the user’s control.

Mastodon has almost no regards to ActivityPub interoperability

Despite being the poster child of the Fediverse and arguably the AP-system with the most users, the implementation of Mastodon lacks in support for common Activity Types, deviates from the standard whenever following it could be a problem for its internal implementation and is known for discarding or ignoring messages from other software when they are not in the “happy path” of Mastodon implementation. For example, Mastodon still lacks good support for groups, which makes it very difficult to follow a Lemmy community from it.

The strongest signal that ActivityPub is a second-class citizen for Mastodon is the fact its clients - from its own web UI to mobile apps done for other Fediverse projects - prefer to use the HTTP API provided by the server instead of accessing ActivityPub’s “native” endpoints.

Mastodon is, first and foremost, an open source clone of Twitter. It was designed to work around the idea of short text messages, with the occasional embedded link or image. If you are a video streamer who wants to share your content, the people on Mastodon will see it as a “post” with a link to the stream. If you write software that shares the music you’d be listening to and you were planning to use it as some alterantive to the “audio scrobblers” like Last.FM, every song would become a post in a stream. There is no way for a user in Mastodon to say “feel free to ignore these types of activities”, because basically every activity gets converted to a “Note” type.

Big instances, big problems

One of the most common analogies for federated services is “it works like email. It doesn’t matter on what server you are, you can talk with anyone who is connected to it”. The analogy works well until you find out that for one reason or another your server got involved in some blocklist. Or your server admin got into a fight with some other admin and they severed ties and took their entire communities as hostages. Or some Japanese script kiddies got an account at your usually well-managed server and is using it to send spam messages to everyone, and other admins now are treating your instance as radioactive.

One unfortunate consequence of the “one server = one domain” equivalence of the current services is that there is no way for someone running the server to separate its users. The whole server needs to be constantly monitored and any slip from the moderation team (which are most of the time just a bunch of overloaded volunteers) could lead to its reputation being destroyed.

To sum up: there was no actual need for yet-another Mastodon instance on the Fediverse. There was nothing novel about it to bring new people to it and by putting energy into this top-down approach was a misfire.

In the next blog post, I’ll do my best to show I’m not a Negative Nancy, and we’ll make a quick dive into ActivityPub and see if by understanding some of the fundamentals we can come up with an alternative approach with better chances of success.

Funkwhale might be the most overlooked Fediverse project

Raphael Lullis — Wed, 10 Apr 2024 20:23:46 +0200

For all the talk about Fediverse alternatives to the platforms from Big Tech, it seems that Funkwhale would be the easiest one to get people out of Spotify/Apple Music/ Youtube Music/etc, and I am trying to figure out what is missing to get more people interested and using it.

I think that its main problem is that no instance provides a “public” catalog of songs that can be streamed/downloaded by its users, but its “social aspect” is really cool. You can, e.g, connect with your friends and choose to share your private libraries with each other. So if you have a good mix of friends who kept/ripped their CD/Vynil collection or bought songs from their favorite indie musicians, you can end up with a pretty extensive library. This makes it a decent (and legal) alternative to sneaker-net piracy.

It also seems like a good alternative for indie labels who want to promote their artists. Set up an instance, get accounts for each artists and let them choose if they want to have their catalog available to the public (or only for paying customers, or give an account for everyone that has bought at least one album from their artists, etc). All of that could be done, it doesn’t really cost a lot to operate and it would make them less dependent on Spotify and/or Bandcamp.

The most interesting thing though, it seems to me that, unlike Lemmy, Mastodon and Pixelfed, a music streaming instance does not depend on a “boil the oceans” approach where the platform is only useful if it reaches a substantially large amount of users.

Thoughts? Why is it that Funkwhale is not being more talked about? Have you tried it? What is missing in your opinion?

Community is not enough

Raphael Lullis — Wed, 07 Dec 2022 01:45:30 +0100

(This post is based on a discussion from Hacker News)

I will assume that the reader is already aware of the mass-exodus from Twitter users to Mastodon, I will also assume that the reader has some basic understanding of how Mastodon works - it is decentralized like email, different servers can talk with each other, etc, etc. What might be news to the reader is the story of how this affects me, given that for the past 3 years I am running a commercial provider for Mastodon.

One can imagine how life-changing it was to see all the news about people looking at the Mastodon as an alternative. With zero marketing, this service that has been used mostly by me/my friends/my family started getting actual signups from strangers from all around the world. The whole system had more activity in the past 3 weeks than in the past 3 years combined. It was incredibly exciting. Many “It’s happening!!!” gifs were sent to my friends on account of each new deposit from customers. It wasn’t only my fortune that seems to have changed: most of the well-known Mastodon providers simply closed to new registrations as they could not keep up with the demand.

I simply couldn’t let go of this opportunity. I gave notice at my job so that I could have time again to work on it. My reasoning was simple: with hundreds of thousands of people leaving Twitter, one would guess that a reasonable percentage of them people would be willing to pay a few dollars per year to get an ad-free, privacy-focused alternative to social media. I guessed that people would be more than willing to pay a few dollars per year to be free from bots, to have a service that felt more “local” and to support smaller, independent service providers instead of fueling the huge corporate machine of BigTech.

Well, it’s been about a month since the initial euphoria, and now it seems that those guesses are becoming second-guessing.

Yes, I have been getting inquiries and signups every day. I need more than two hands to count the number of paying customers, but where are the rest of the hundreds of thousands of people going?

Turns out, they are going to the handful of “free” instances that have sprung up to fill the demand. Some of them got popular enough to attract celebrities (oh, my!), others have grown so fast that they ended up facing issues and got a nice war story to tell.

My first reaction to these stories were admittedly filled with a bit of envy. “Wow, 30000 users! If that many people subscribed to my $0.50/user/month package, it would be enough to pay myself a full salary!”. But then I realized that they are only getting these many people because they are not driven by commercial interests: even with donations, I am sure they are not collecting enough to keep things afloat and they keep going because they don’t mind spending all this time, money and resources of their own on this project. They can treat it as a (relatively expensive) hobby, and they can keep it running as long as it satisfies them.

The problem I see here is that I think the approach is short-sighted and harmful in the long run. Yes, people now are finally starting to see the issue with ad-funded social media, and it is good that this first wave can find a home at a “free” instance. But If we want to provide an ethical alternative to Twitter/Facebook/Instagram/WhatsApp, we need to find a way to serve hundreds of millions to billions of users. How are these people going to be spread around the instances?

Are we going to create ~15000 Hachyderms to replace Twitter?

Are we going to create ~60000 pixelfeds to replace Instagram?

Are all these communities depend on volunteers to do the grunt work?

Are all these instances going to be operated by professionals from Silicon Valley who can take some time from their cushy jobs at unethical companies and are able to afford to sink a few hundred dollars every month?

I don’t think so. I think this approach is not sustainable in the long term. I think that eventually, all these “free” instances will find out that they need to start thinking like a business to keep the lights on. From there, what is going to happen? Are they going to go to appeal to “your donation is very important to us” and expect that a few generous souls make up for the free-riders? Or are they going to pull a bait-and-switch, and start charging for access from all users? Or even worse: are they going to copy the VC-style playbook, focusing on growing the user base first at all costs and figuring out monetization later?

Word of warning: this was what got us in this mess in the first place.

If we really want to get rid of the evils of Big Tech and if we want social media companies to serve the people and not the interests of the elites, we need to understand TANSTAAFL. We need to understand that not all commercial providers are evil capitalist pigs, and we need to understand that we will be better off by paying a fair, small amount every month instead of relying on crazy schemes that try to hide reality: for this to become accessible to everyone, real money needs to be spent on the people working on this and on the servers that are running 24/7 to store and distribute the hot takes and stupid memes that we so bizarrely crave every day.

I worry that if we don’t change the mindset quickly, the whole Twitter drama would be a wasted opportunity and we will go back to the status quo, where surveillance capitalism is the norm and truly open systems like Mastodon are just a geeky curiosity.

Let's fend off Big Tech and find a way to fund FOSS we all can be fond of

Raphael Lullis — Fri, 06 May 2022 12:22:34 +0200

Here is an incomplete, completely half-baked idea that I have been chewing on since Filippo’s call to professionalize the work of OSS maintainers. I want to share it here because (1) I think there is a good business opportunity here that could use some validation and (2) I am both an OSS project maintainer that would like to be paid for my work and I also have a business that depends on having quality open-source solutions.

The idea is to build a system where:

individuals are incentivized to contribute financially to OSS.
individuals can also promote their OSS work, without getting into a popularity contest.
individuals can collaborate on teams, who are self-governing.
companies can help contribute to many teams at the same time, without having to deal with complicated bureaucracy. Ideally they would pay one single invoice, and it would look like a service that they are paying.

Ok, are you ready?

Wait! Don’t leave. Please hear me out! I know that at first is going to sound a lot like LinkedIn, but I think we can make some twists and get something useful out of it.

We’d start with all the standard features from a social network: a profile page, a way to write posts, follow others, participate in discussions, etc. Being a social network focused on FOSS, let’s also imagine a feature where every user can have a page for their projects and that people can organize themselves in teams.

Still too much like LinkedIn? Let’s also consider a Patreon-style sponsorship program. Any team or project can offer special rewards or exclusive products for other users who back them directly.

Ok, it is still pretty standard. But let’s add the first couple of twists:

access to the website is obtained through a membership of $3-5/month.
all revenue from users is earmarked for the “Developers Fund”, which is paid out on a monthly basis to all teams based on Quadratic Voting

You will need millions of users just to get any decent revenue, you are going to say… which gets to second leg of this monster we are creating.

A suite of services for companies

Companies would certainly be interested in having access to this potential qualified userbase. There are a good amount of services that could be provided for them.

A Job board. To keep with the OSS theme, we could make it that companies could pay a $ X00 month to post as many OSS jobs they want, or they could pay twice as much individual non-OSS job listing. Now that StackOverflow has moved out of the job listing market, there is some room for such an offering.
Ads. Companies could be interested in promoting their products, developer conferences, run a focus group, etc. As long as the ads are not privacy-invasive, they are acceptable.

There is also nothing stopping companies to use the platform to create grant programs, become “official sponsors” of a project or a team or create hackathons that pay meaningful prizes.

The important thing however is that it is the revenue from the services paid by the companies is what can turn this into a proper business. Of course part of this revenue could (and should) go to the Developers Fund, but my guess is that if you keep it lean your operating costs can be quite low. There is also the chance of turning such a venture into a proper 503 Foundation, which could not only bring tax benefits but also become an alternative to those who are tired of the FSF current actions and outdated leadership.

Let me know your thoughts

Like I said, these are just some ideas that I have been trying to develop further. You can comment on Hacker News and I hope to update this page with anything that comes up.

Thinking heads are not in the clouds

Raphael Lullis — Mon, 08 Nov 2021 00:43:25 +0100

Part I - Can it run OpenOffice?

My first job in Germany was for this T-Labs project: an innovative online-first, browser-based operating system which promised to make it simple for enterprises to manage the devices for their mobile workforce. The OS was based on Linux and Firefox and it offered a pretty standard set of features: users would have an SSO system and login directly to their office intranet through a VPN connection where they would get access to their web applications, base software updates would be delivered transparently; and any on the device would be encrypted. In theory, even if someone lost their device they could be re-issued a new one and continue their work without any worry.

We’ve built quite a lot of interesting components for that: custom Firefox profiles to substitute Linux login screens, specialized network managers that ensured users always had the best connection and didn’t need to know anything about its setup, extensions to display widgets that would usually be provided by base desktop environment (e.g, clock, network status, mounted usb), etc.

The project was already over an year in development after it got one interested customer. After the demo, they were enthusiastic about the product and its direction, but they had one important requirement:

Can it run OpenOffice?

All of the “secure access to the intranet” was useful, but they already had re-trained their workforce when they moved from MS-Office to OpenOffice, they were not interested in doing that again. Their intranet could give them access to the files, but the real work would be done at the device running OpenOffice.

When talking with their tech people, They pressed our management a bit: what happens if the user wants to upload a file and they don’t have internet access? Oh, the file gets to the local filesystem and is later synchronized, then why not do like Dropbox and have all documents in the local filesystem?! Then you could also run OpenOffice!

Part II - KISS’ing my entire contact list (management)

In my quest to de-google my digital life, I was looking for an alternative to Google Contacts, specifically one that was easy to use with my de-googled Android phone. Being no stranger to self-hosting services, I started to look into what were the most commonly used alternatives and found that Nextcloud provided a CalDav/CardDav server. A few docker-compose commands later and I had it nicely running on my home server.

Something didn’t sit right with the idea, though. Nextcloud does a lot of things that can substitute Google services, but would I even use them? All I cared about was my contact list. I already have an email provider with its own calendar server, I don’t use Google Docs and most of my “work documents” are git repositories - which by its very nature are easy to synchronize between computers. Sure, I’ve also later found radicale, which seems to be focused only on being Cal/CardDav server, but did I really need a whole web application just to keep some files in sync between my different devices?

The answer: of course servers are not needed, you can sync the files in a structured format which will then then allow your Android apps to work as a contact provider.

So now my phone may have 99 problems but at least contact management ain’t one.

Part III - Stream me not.

How does a stream-service-denier listens to music in 2021?

This is what I would do for my music collection of ~10000 songs and ~175GB of high-quality mp3 and FLAC files in different ways:

When at home, I connect to the NFS mount on my NAS that have all the music files and I can play with my local media player.
My laptop has a large-ish SSD, so I can have a music folder managed with Syncthing and all the songs are also there for me to listen.
For my phone, I’d upload my music to Google so that I could choose what to download to my phone later… and then I wake up in a jolt, realizing Google killed once-again a service that was useful.

Oh, well… I wanted to de-google anyway. So let’s look at the alternatives:

Subsonic - “Your complete, personal media server”
Ampache - Music Streaming server
Jellyfin - enables you to collect, manage, and stream your media.
Funkwhale - A social platform to enjoy and share music

Noticing a pattern?

I ended up installing Jellyfin and Funkwhale. Funkwhale claimed support for Subsonic API anyway and Jellyfin also had some android clients. As long as these clients had offline support and worked similarly to Google Music, I would be satisfied again.

Did it work?

Unfortunately, no. I’ve tried three different Subsonic players for Android which claimed offline playing ability along with the jellyfin client. None of them were able to save the downloaded songs to external storage, even with all relevant permissions granted. They only worked with internal storage and/or when using their own player. Songs that were downloaded by any one client would not be visible to an external media player.

So, we have four different projects all working on their own separate nice web applications, when all I really wanted was to run ~~OpenOffice~~ Metro?.

Everything is (still) a file.

The common thread around all experiences above: the premise that we need to have applications running on the cloud is broken. We accepted this because it made it easier for developers to distribute their software, not because their software is necessarily better for us.

What we need is to have a device-transparent way to see our data. We got so used to the idea that web applications let us work from “dumb terminals” that we failed to realize that there is no such thing as a dumb terminal anymore. With multi-core smartphones, many of them with 4, 8, 12, 16GB of RAM; it’s not too hard to notice that the actual bottlenecks in mobile devices are battery life and (intermittent and still relatively expensive) network connectivity. These are problems that can be solved by appropriate data synchronization, not by removing the data from the edge.

One of the early jokes about web2.0 was that to have a successful company you should take an Unix utility and turn it into a web app. This generation of open source developers are reacting to this by looking at successful companies and building “self-hosted” versions of these web apps. What they didn’t seem to realize is that we don’t need them. The utlities and the applications still work just fine, we just need to manage the data and how to sync between our mobile/edge devices and our main data storage.

If you are an open source developer and you are thinking of creating a web app, do us all a favor and ask yourself first: do I need to create yet-another silo or can I solve this with Syncthing?

Can FOSS eat the world?

Raphael Lullis — Tue, 26 Oct 2021 17:45:25 +0200

Yesterday I got the first (paid!) request to add a feature on Hub20 from someone who is working on what is likely to become the first Hub20 production instance.

It’s a simple feature, driven by his need to have some functionality that is not provided yet. He could’ve looked at current state of the work and said “oh, it sucks because it doesn’t do X”. He could’ve used that (temporary) failure to address his need, and justify going to some “established” business. Or he could’ve just taken the code and tried to “compete” - it’s open source after all.

Instead, he decided to look at things that Hub20 does do, and chose to support it to make it better.

Even if there were a closed proprietary service that could do what he needed (I have looked, and I haven’t found it yet), his contribution will ensure that his business is not so dependent on someone else’s. And if tried to compete with Hub20, he would quickly realize the time alone get his development environment in order was better spent working on his own business.

Most interesting of all is that his contribution (while valuable) would never buy the amount of “resources” if he were trying to do this in-house. If I were to charge regular consulting rates to have this done only for his business, it would have cost him 3-5x more to have something that would end up probably being of worse quality and that would have no chance to be further improved without having to spend again.

Could it be that bigger companies will learn this lesson and start operating on this model as well? For all the talk about how hard it is to find good/talented people in the market, why not reduce the size and scope of your development team to your absolute core functions, and “outsource” anything possible to open source projects?

Some companies are already proving this is possible. The question is what is stopping more of them from doing so. Does it look bad from an accounting perspective, so the bean-counters twist their noses at the idea? Is it too much of a cultural shift, and if so shouldn’t we take advantage of the fact that we are already in the middle of a massive cultural shift and push this one along? Is it a matter of corporate politics? Would a Team Lead/Manager/VP be shooting their careers down by advocating their team to be smaller and to “give money away”?

A proposal for a decentralized BAT cartel

Raphael Lullis — Mon, 30 Aug 2021 01:46:40 +0200

Preface

Due to my (short) time working on Raiden and (currently moonlighting) at Hub20, I got quite a bit familiar with the different crypto projects that were coming with the “DeFI” wave. Something in common to most of these projects is the fact that they don’t really have any idea where their actual money will come from. Being a witness to the emergence of “DAO governance” tokens that pay absolutely zero dividends to ideas based on bad economic principles and outright scams, I started to develop a good sense of what projects were building any kind of actual wealth and could be considered proper investments and which were just playing some sort of ponzi game.

Out of the “good” crypto projects, there is one that started even before the DeFI craze, in 2017: the Basic Attention Token from Brave.

BAT and Brave, quickly explained

Brave is a web browser that focuses on privacy and protecting users from data tracking and the exploitation of personal data made by Big Tech. It offers a built-in ad blocker, blocks third-party tracking cookies and does a lot more.

Brave’s business model is based on the idea that they can offer advertisements that don’t depend on exploiting personal data. Users from the browser can opt in to receive ads (as system notifications) which are from Brave’s ad network. Users participating in the program receive a share of 70% of all advertisements they receive. These payments are made using BAT, a cryptocurrency that is based on the Ethereum blockchain.

BAT is a token that has fixed, limited supply. During the ICO, it was established that a total of 1.5 billion BAT would be minted, divisible in up to 18 decimal points.

How much is the price of BAT affected by Brave’s growth?

It’s important to notice: Brave’s revenue is in USD, the payouts are in BAT. Advertisers buy space in the inventory from Brave, and Brave takes part of that money to buy BAT in the open market. This means that the price of the token has no direct influence in Brave’s revenue or in the dollar amount from the payouts.

At the moment, Brave’s ad rewards requires ~0.1% of the whole BAT market cap (~1.25 billion USD), so these monthly buys are not enough to cause any kind of noticeable change in BAT’s price. The price of token is mostly determined by speculation and oscillates with the rest of the market - notably BTC.

Solving the lack of utility and overabundance: a crypto-cartel.

BAT has in its favor the fact that it has at least one buyer that is always going to be interested in acquiring the tokens regardless of its price. However, the supply is too large for the overall economy. Given that there is no easy way to increase the demand, perhaps we can think of appoaches that reduce the supply?

MakerDAO Vaults

At first, my idea was around using another solid project from the DeFI world, MakerDAO. MakerDAO has built the first decentralized stablecurrency on ethereum, called DAI. It is based on a set of smart contracts that takes some volatile crypto asset and provides collaterized loans in DAI to users.

BAT is one of the few tokens that is accepted by MakerDAO, and while there is a whole other post that I could write about the advantages of using stabletokens, let’s just point out that more people creating BAT vaults would lead to a reduction in BAT’s circulation.

However, it is a collaterized loan, which means that it costs to leave the money locked, and if the price of the token oscillates too much, it may lead to users losing their funds.

Improving on the idea: lock BAT, make withdrawal dependent on Brave’s revenue

The next idea has no existing implementation, but (I believe) could be developed by any reasonably experienced smart contract developer. Like MakerDAO, it takes the idea of locking BAT into a vault and it would also lead to users having stabletokens as the “usable” currency.

Such protocol would be based on a smart contract with the following functionality:

Anyone can deposit BAT to be controlled by the contract, which would then lock these funds.
The smart contract would then mint another token which would represent the ownership of a share of the funds locked by the contract.
Every month, the contract would unlock (allow for withdraw) an amount of BAT corresponding to 70% of Brave’s reported revenue from their ad distribution. To withdraw the allowed BAT, the contract would accept stabletokens (e.g, DAI/USDC)
The received stabletokens would then be redeemed by the BAT depositors.

In effect, every one locking their BAT would be giving away their chance of making a profit with BAT speculation, but would receive a perpetual dividend proportional to Brave’s revenue.

Brave has been growing at a good pace and it seems to be a safe bet in the sense that it will be around in the coming years. Those that want to look at defi as serious investment vehicle can easily be interested in this system where the risks are relatively low and that can lead to a continuous cash stream.

Loopring Arb checker - or trading hours of fun work to save on minutes of tedious calculation

Raphael Lullis — Tue, 16 Feb 2021 23:39:20 +0100

While I’m waiting for the next release of raiden so that I can finish my own release of Hub20, I am left playing with other layer-2 projects. Of those, loopring is certainly the one that has caught most of my attention. It requires people to transfer their tokens to a contract and then lets you keep a “Layer-2 balance”.

Loopring and the mechanics of an AMM

Any transaction that happens on Layer-2 is virtually free, and holding the tokens there cost nothing. Whether you need to send a payment to another Layer-2 account or even back to a different Layer-1, it can be done at much lower cost than a standard on-chain transfer.

That alone should make it interesting, but to make it even more appealing the folks at Loopring created an Automated Market Maker (AMM) exchange: similar to the mechanism popularized by Uniswap, users can provide liquidity in token pairs, and users that want to exchange one token for another (make a swap) can do so at a fee.

Due to the way that an AMM exchange works, it can happen to become unbalanced, i.e: if users buy too much of one side of the pool or if one token of the pair gains/loses value relative to the other, the value in one side of the pool is different from the other. This in turn creates an opportunity for arbitrage: one can buy the token on the side with “excess” supply for a lower price than other pools. In the process, the pool gets re-balanced and peace is restored in the universe - that is at last until the next trader makes it unbalanced again and the dance starts all over.

Enough exposition. How can I make money from it?

When Loopring started accepting more token pairs to be listed on the exchange,they rely on users providing lots of liquidity of all the different tokens. They started providing rewards for people providing liquidity for some tokens that were of interest, but not all pools were benefiting from these incentives.

As I noticed that there were some tokens from solid projects but with low liquidity, I wondered if I could check if how balanced those pools were.

After some hours playing with their API, I hacked up a script that checks that takes a list of tokens available for trade and does the following:

Checks if there is a “ETH-” pair
Verifies the volume and USD value of each side of token pair
In case the pool is unbalanced, calculates how much of any one token is needed to rebalance the pool and the associated costs.
If the swap is profitable, prints the result with instructions of what token and how much to trade.

The code is available here and is a simple python script. To use it, just clone the repo, pip install the dependencies listed on requirements.txt and call python loopring.py. An example run would be:

Pay 0.2878332608336053 ETH ($511.2580728904747)
Receive: 190.44220898248264 CRV ($523.7160747018272)
Profit: $12.458 (2.437%)

[<Token: ETH>, <Token: DAI>] is balanced

Pay 0.07103439215219157 ETH ($126.17341837248723)
Receive: 61.38390289979394 GRT ($126.45083997357551)
Profit: $0.277 (0.220%)

[<Token: MKR>, <Token: ETH>] is balanced

[<Token: ETH>, <Token: USDC>] is balanced

Pay 87.2081188782998 ETH ($154901.67699520246)
Receive: 223365.22777104643 LRC ($155110.62166013668)
Profit: $208.945 (0.135%)

[<Token: WBTC>, <Token: ETH>] is balanced

[<Token: LINK>, <Token: ETH>] is balanced

Pay 42.64371343612116 1INCH ($201.7047645528531)
Receive: 0.11383943982241368 ETH ($202.20502819576586)
Profit: $0.500 (0.248%)

Pay 35.65256489452435 UNI ($724.81664430568)
Receive: 0.4083333892596485 ETH ($725.2940160046655)
Profit: $0.477 (0.066%)

Pay 0.3363907664508281 COMP ($149.6198851019993)
Receive: 0.08453349180284586 ETH ($150.1509241449689)
Profit: $0.531 (0.355%)

Pay 2683.6068944875324 PNK ($241.51925329008893)
Receive: 0.1372335288461172 ETH ($243.75831094233874)
Profit: $2.239 (0.927%)

[<Token: RENBTC>, <Token: ETH>] is balanced

Pay 4.064107695371856 AAVE ($1727.3676937638998)
Receive: 1.0481508187995057 ETH ($1861.7569288762459)
Profit: $134.389 (7.780%)

Taking a look at the results, it’s easy to verify that the ETH-AAVE pool is really unbalanced and is now a big opportunity to make some quick cash. As loopring grows, more people will come in, more sophisticated traders will show up and these opportunities will become ever rarer. Enjoy it while it lasts!

Federations and Identity

Raphael Lullis — Thu, 30 Jul 2020 16:29:18 +0200

Now that Communick is available for the public at large and I am looking for ways to spread the word about the benefits of decentralized services, something very curious is happening: people seem to understand the issue of centralized services, they understand how federation works and that they can follow people from anywhere, yet the most common question I am hearing is “What instance should I join if I am interested in X? Which ones are better for people talking about Y?”.

My response would be something along the lines of “Do you use a different phone number or email address depending on the topic of the conversation?”, but the blank stares quickly remind me that I am part of the last generation that actually talked on their phones and wrote emails to actual people.

Yes, I am becoming an old fart, but I don’t know if it is just a generational divide. My hunch is that companies were very successful in convincing people that they are what they consume. As well meaning their offering can be, every new social media startup that pops up wants to become a middleman in the conversation of an insular tribe. To have any kind of conversation, they need to have people joining in the network, and playing to people’s sense of identity is one of the oldest marketing tricks¹ to help them grow their usage numbers. This is already bad in itself, and the fact that they target younger people - who are still in a process of self-discovery and understanding their relationships with society at large - only makes it worse. It is going to take some time to get people to unlearn this and to stop conflating their peer group values with their own.

Back to the idea of ActivityPub and Mastodon specifically, this conflation of instances with tribes maybe made sense in a initial point, where its participants did not have any way to find each other. Content discovery in the fediverse certainly still needs to improve in order to make it easier for newcomers and those that want to leave centralized networks, but keeping this focus on “topic-specific” instances is a recipe to keep a great idea forever tainted as something for fringe groups.

Outside of these cases where the identity of the individual is tied to membership of the group, people’s main focus when selecting an instance should have nothing to do with the group: is the service reliable? Is the moderation fair and consistent? Is user privacy being respected? Do they follow good security practices when running the service?

There certainly is a place for instances whose members have some shared identity: employees of a business that need to communicate on behalf of the company, journalists and reporters writing for a specific news group², and even those that want to express some currently-taboo aspect of their personalities (Yes, Uncle John, keep your chiauauas-dressed-as-sailor-moon hentai collection private, please) will be better off using accounts from a group specific instance. Most of us will however be better served by joining a a neutral federation or - even better - by running the instance under your own domain.

It is no wonder that the current populist wave is tied to social media, where is so easy to foment the us-vs-them mentality for pretty much any topic, is it? ↩︎
Given all the talk about how media groups are at odds with big tech, how come not a single one of the mainstream news groups put together their own instance of federated service yet? It’s almost like they want to have all of their brand value thrown away and mixed with all the bots and shitposters of Twitter and Facebook. ↩︎