(This is a repost of this reddit post https://www.reddit.com/r/selfhosted/comments/1fbv41n/what_are_the_things_that_makes_a_selfhostable/, I wanna ask this here just in case folks in this community also have some thoughts about it)

What are the things that makes a selfhostable app/project project good? Maybe another way to phrase this question is, what are the things that makes a project easier to self-host?

I have been developing an application that focuses on being easy to selfhost. I have been looking around for existing and already good project such as paperless-ngx, Immich, etc.

From what I gather the most important thing are:

  • Good docs, this is probably the most important. The developer must document how to self-host
  • Less runtime dependency–I’m not sure about this one, but the less it depends on other services the better
  • Optional OIDC–I’m even less sure about this one, and I’m also not sure about implementing this feature on my own app as it’s difficult to develop. It seems that after reading this subreddit/community, I concluded that lots of people here prefer to separate identity/user pool and app service. This means running a separate service for authentication and authorization.

What do you think? Another question is, are there any more good project that can be used as a good example of selfhostable app?

Thank you


Some redditors responded on the post:

  • easy to install, try, and configure with sane defaults
  • availabiity of image on dockerhub
  • screenshots
  • good GUI

I also came across this comment from Hacker News lately, and I think about it a lot

https://news.ycombinator.com/item?id=40523806

This is what self-hosted software should be. An app, self-contained, (essentially) a single file with minimal dependencies.

Not something so complex that it requires docker. Not something that requires you to install a separate database. Not something that depends on redis and other external services.

I’ve turned down many self-hosted options due to the complexity of the setup and maintenance.

Do you agree with this?

  • sylver_dragon@lemmy.world
    link
    fedilink
    English
    arrow-up
    12
    arrow-down
    1
    ·
    1 month ago

    My list of items I look for:

    • A docker image is available. Not some sort of make or build script which make gods know what changes to my system, even if the end result is a docker image. Just have a docker image out on Dockerhub or a Dockerfile as part of the project. A docker-compose.yaml file is a nice bonus.
    • Two factor auth. I understand this is hard, but if you are actually building something you want people to seriously use, it needs to be seriously secured. Bonus points for working with my YubiKey.
    • Good authentication logging. I may be an outlier on this one, but I actually look at the audit logs for my services. Having a log of authentication activity (successes and failures) is important to me. I use both fail2ban to block off IPs which get up to any fuckery and I manually blackhole entire ASNs when it seems they are sourcing a lot of attacks. Give me timestamps (in ISO8601 format, all other formats are wrong), IP address, username, success or failure (as a independent field, not buried in a message or other string) and any client information you can (e.g. User-Agent strings).
    • Good error logging. Look, I kinda suck, I’m gonna break stuff. When I do, it’s nice to have solid logging giving me an idea of what I broke and to provide a standardized error code to search on. It also means that, when I give up and post it as an issue to your github page, I can provide you with some useful context.

    As for that hackernews response, I’d categorically disagree with most of it.

    An app, self-contained, (essentially) a single file with minimal dependencies.

    Ya…no. Complex stuff is complex. And a lot of good stuff is complex. My main, self-hosted app is NextCloud. Trying to run that as some monolithic app would be brain-dead stupid. Just for the sake of maintainability, it is going to need to be a fairly sprawling list of files and folders. And it’s going to be dependent on some sort of web server software. And that is a very good place to NOT roll your own. Good web server software is hard, secure web server software is damn near impossible. Let the large projects (Apache/Nginx) handle that bit for you.

    Not something so complex that it requires docker.

    “Requires docker” may be a bit much. But, there is a reason people like to containerize stuff, it avoids a lot of problems. And supporting whatever random setup people have just sucks. I can understand just putting a project out as a container and telling people to fuck off with their magical snowflake setup. There is a reason flatpak is gaining popularity.
    Honestly, I see docker as a way to reduce complexity in my setup. I don’t have to worry about dependencies or having the right version of some library on my OS. I don’t worry about different apps needing different versions of the same library. I don’t need to maintain different virtual python environments for different apps. The containers “just work”. Hell, I regularly dockerize dedicated game servers just for my wife and I to play on.

    Not something that requires you to install a separate database.

    Oh goodie, let’s all create our own database formats and re-learn the lessons of the '90s about how hard databases actually are! No really, fuck off with that noise. If your app needs a small database backend, maybe try SQLite. But, some things just need a real database. And as with web servers, rolling your own is usually a bad plan.

    Not something that depends on redis and other external services.

    Again, sometimes you just need to have certain functionality and there is no point re-inventing the wheel every time. Breaking those discrete things out into other microservices can make sense. Sure, this means you are now beholden to everything that other service does; but, your app will never be an island. You are always going to be using libraries that other people wrote. Just try to avoid too much sprawl. Every dependency you spin up means your users are now maintaining an extra application. And you should probably build a bit of checking into your app to ensure that those dependencies are in sync. It really sucks to upgrade a service and have it fail, only to discover that one of it’s dependencies needed to be upgraded manually first, and now the whole thing is corrupt and needs to be restored from backup. Yes, users should read the release notes, they never do.
    The corollary here is to be careful about setting your users up for a supply chain attack. Every dependency or external library you add is one more place for your application to be attacked. And just because the actual vulnerability is in SomeCoolLib.js, it’s still your app getting hacked. You chose that library, you’re now beholden to everything it gets wrong.

    At the end of it all, I’d say the best app to write is the one you are interested in writing. The internet is littered with lots of good intentions and interesting starts. There is a lot less software which is actually feature complete and useful. If you lose interest, because you are so busy trying to please a whole bunch of idiots on the other side of the internet, you will never actually release anything. You do you, and fuck all the haters. If what you put out is interesting and useful, us users will show up and figure out how to use it. We’ll also bitch and moan, no matter how great your app is. It’s what users do. Do listen, feedback is useful. But, also remember that opinions are like assholes: everyone has one, and most of them stink.

  • Matt@lemmy.ml
    link
    fedilink
    English
    arrow-up
    8
    ·
    edit-2
    1 month ago

    Not something so complex that it requires a docker

    Docker is the thing that sandboxes your services from the host OS. I’d rather use Podman because of the true non-root mode, but Docker is still based. Plus, you can use Docker Swarm if you don’t want to switch to Kubernetes (though you don’t have easy storage integration for persistence).

    • daddy32@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 month ago

      Docker is also the thing that allows the distribution of the app as “single file with minimal dependencies”.

  • bandwidthcrisis@lemmy.world
    link
    fedilink
    English
    arrow-up
    5
    ·
    1 month ago

    Before even getting to documentation, I see so many projects that don’t have a short summary of what they do (and maybe what to not expect them to do).

    As an example, Home Assistant. I can tell that it involves home automation, so can I replace Google Home with it? It seems like it doesn’t do voice recognition without add-ons and it can work with Google Assistant. Do I still need accounts with the providers of smart appliances, or can it control my bulbs directly?

    None of that is very clear from the website.

    I’ve seen plenty of other projects where it’s assumed there’s no need to explain it’s overall purpose.

  • 𝘋𝘪𝘳𝘬@lemmy.ml
    link
    fedilink
    English
    arrow-up
    5
    arrow-down
    1
    ·
    1 month ago

    To me the number one thing is, that it is easy to setup via Docker. One container, one network (ideally no network but just using the default one), one storage volume, no additional manual configuration when composing the container.

    No, I don’t want a second container for a database. No I don’t want to set up multiple networks. Yes, I already have a reverse proxy doing the routing and certificates. No, I don’t need 3 volumes for just one application.

    Please just don’t clutter my environment.

    • hono4kami@slrpnk.netOP
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 month ago

      No, I don’t want a second container for a database.

      Unless you’re talking about using SQLite:

      Isn’t the point of Docker container is to only have one software/process running? I’m sure you can use something like s6 or other lightweight supervisor, but I feel like that’s seems counterintuitive?

      • 𝘋𝘪𝘳𝘬@lemmy.ml
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 month ago

        To me, the point of Docker is having one container for one specific application. And I see the database as part of the application. As well as all other things needed to run that application.

        Since we’re here, lets take Lemmy for example. It wants 6 different containers with a total of 7 different volumes (and I need to manually download and edit multiple files before even touching anything Docker-related).

        In the end I have lemmy, lemmy-ui, pictrs, postgres, postfix-relay, and an additional reverse proxy for one single application (Lemmy). I do not want or need or use any of the containers for anything else except Lemmy.

        There are a lot of other applications that want me to install a database container, a reverse proxy, and the actual application container, where I will never ever need, or want, or use any of the additional containers for anything else except this one application.

        So in the end I have a dozen of containers and the same amount of volumes just to run 2-3 applications, causing a metric shit-ton of maintenance effort and update time.

        • Zeoic@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          ·
          1 month ago

          I agree with this. If you are going to be using multiple containers for a single app anyways, what is the point of it being in multiple containers? Stick all of it in one container and save everyone the hassle.

    • ÚwÙ-Passwort@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 month ago

      I prefer this, but if the options are available its shows me that soneone actually thought about it while creating the software/conatiner

  • T156@lemmy.world
    link
    fedilink
    English
    arrow-up
    4
    ·
    1 month ago

    Ease of installation/use, I think, is the main big one, and one of the biggest obstacles.

    People who want to give self-hosting a try aren’t going to be particularly fond of having to jump through a whole bunch of different configs, and manually set everything up.

    They want something that they can just set up and go, without having to deal with server hosting, services, and all of that. Something you can just run on your computer, leave it be, and use it with relatively little fuss.


    Second to that, would definitely be a case of better documentation/screenshots. A lot of self-hosted things, like Lemmy, didn’t provide much documentation of what the actual user side of it does, only what you need to do to set it up, which isn’t going to make me want to use the software, if I have no idea what it’s supposed to do, and how it compares to other things that do the same.

  • thelittleblackbird@lemmy.world
    link
    fedilink
    English
    arrow-up
    3
    ·
    1 month ago

    My points are totally in the other direction:

    • stable, this is critic, if the app is not able to performs its duties with. 2 weeks uptime, then it is bad. This also applies to random failures. I don’t want to spend endless days to fix it
    • docker, with a all-in-image, and as a nice to have the possibility to connect external docker composes for vpn, or databases
    • a moderate use of resources, not super critic, but nobody likes to have ram problems

    And then as a second league that lean the balance:

    • integration with LDAP or any central user repo
    • relatively easy to backup and restore
    • relatively low level of break changes from version to version
    • the gui / ease of use (in like with the complexity of the problem I want to address)
    • sane use of defaults and logging capabilities

    That’s all from my side

  • e0qdk@reddthat.com
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 month ago

    Do you agree with this?

    Yes, at least for hobby use. If it really needs something more complex than SQLite and an embedded HTTP server, it’s probably going to turn into a second job to keep it working properly.

  • Prunebutt@slrpnk.net
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 month ago

    Please be mindful of HDD spindown.

    If your app frequently looks up stuff in a database and also has a bunch of files that are accessed on-demand, then please have an option to separate the data-directory from the appdata-directory.

    A lot of stuff is self-hosted in homes and not everyone has the luxury of a dedicated server room.

    • hono4kami@slrpnk.netOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 month ago

      separate the data-directory from the appdata-directory

      Would you mind explaining more about this?

      • Prunebutt@slrpnk.net
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 month ago

        Take my setup for jellyfin as an example: There’s a database located on the SSD and there’s my media library located on an HDD array. The HDD is only spun up when jellyfin wants to access a media file.

        In my previous setup, the nextcloud database was located on a HDD, which resulted in the HDD never spinning down, even if the actual files are never really accessed.

        In immich, I wasn’t able to find out if they have this separation, which is very annoying.

        All this is moot, if you simply offer a tiny service which doesn’t access big files that aren’t stored on SSDs.

  • Passerby6497@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 month ago

    Configuration by config file is preferred but not mandatory for me, but a docker image is mandatory for me to even try the app anymore. And the ability to backup and restore state is key, preferably in such a way that I can write my backup to a mounted smb share rather than writing locally and copying to the network.

    I’m running everything on commodity or 2nd hand gear, so failures aren’t unheard of. I had one of my micro PCs cook itself this year, and the majority of my services on that box fit that mold (mostly), so I got them back up pretty quick. Though, I did run into issues with container backups not working (because they write the backup like a database, so it has to be a local write for a db lock) and had to start from scratch .

  • tatterdemalion@programming.dev
    link
    fedilink
    English
    arrow-up
    2
    ·
    edit-2
    1 month ago
    • Has a simple backup and migration workflow. I recently had to backup and migrate a MediaWiki database. It was pretty smooth but not as simple as it could be. If your data model is spread across RDBMS and file, you need to provide a CLI tool that does the export/import.

    • Easy to run as a systemd service. This is the main criteria for whether it will be easy to create a NixOS module.

    • Has health endpoints for monitoring.

    • Has an admin web UI that surfaces important configuration info.

    • If there are external service dependencies like postgres or redis, then there needs to be a wealth of documentation on how those integrations work. Provide infrastructure as code examples! IME systemd and NixOS modules are very capable of deploying these kinds of distributed systems.

  • InnerScientist@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    edit-2
    1 month ago

    One thing that makes a project good is knowing what it does, I’ve seen quite a few projects where they talk about all the features and technology and how to configure it but not a word about what it actually does, what problems it solves and so on.

    I won’t self host your program if you don’t even tell me what it does, don’t make me search and clue together large parts of the documentation just to find if I want it. A simple explanation is enough but somehow I’ve seen quite a few programs that don’t have it.