12

I just got my home server up and running and was wondering what you guys recommend for backups. I figure it will probably be worth having backups on cloud servers tjay are external, are there any good services yall use for that?

all 37 comments
sorted by: hot top controversial new old
[-] gobbling871@lemmy.world 3 points 1 year ago

Backblaze b2, borgbase.com. There are also programs like dejadup that will let you backup to popular cloud drives. The alternatives are limitless.

[-] vegetaaaaaaa@lemmy.world 2 points 1 year ago
[-] spez_@lemmy.world 2 points 1 year ago* (last edited 1 year ago)

I use Restic + Resticprofile to back up everything and store it on my local HDD.

Then, I use Rclone to sync the local repository to Backblaze B2.

Here's my general setup:

/.config/restic/
├── logs
│   ├── statuses
│   │   ├── restic-status-20230202T020202.json
│   │   └── restic-status-20230101T010101.json
│   ├── restic-check-20230202T020202.log
│   └── restic-backup-20230101T010101.log
├── config
│   ├── profiles.yaml
│   ├── excludes.txt
│   ├── rclone.conf
│   └── password.txt
├── bin
│   ├── restic_0.15.2_linux_arm64
│   ├── rclone_1.63.1_linux_arm64
│   └── resticprofile_0.22.0_linux_arm64
version: "1"

# Schedules (https://www.freedesktop.org/software/systemd/man/systemd.time.html#Calendar%20Events)
{{ $SCHEDULE_RESTIC_BACKUP := "*-*-* 22:00:00" }}       # Daily at 10PM
{{ $SCHEDULE_RESTIC_CHECK := "Sat *-*-* 04:00:00" }}    # Weekly at 4AM on Saturday
{{ $SCHEDULE_SYNC_BACKUP := "Sun *-*-* 21:30:00" }}     # Weekly at 11.30PM on Sunday
{{ $SCHEDULE_POSTGRES_BACKUP := "Fri *-*-* 20:00:00" }} # Weekly at 8PM on Friday

# Directories
{{ $LOCATION_RESTIC_BINARY := "/home/deck/Desktop/.config/restic/bin/restic_0.15.2_linux_arm64" }}
{{ $LOCATION_RESTIC_REPO := "/home/deck/Desktop/restic-repo" }}
{{ $LOCATION_RESTIC_LOG := "/home/deck/Desktop/.config/restic/logs" }}
{{ $LOCATION_RESTIC_STATUS := "/home/deck/Desktop/.config/restic/logs/statuses" }}
{{ $LOCATION_RESTIC_BLOCKED_FILE := "/home/deck/Desktop/.config/restic/BLOCKED" }}
{{ $LOCATION_RCLONE_BINARY := "/home/deck/Desktop/.config/restic/bin/rclone_1.63.1_linux_arm64" }}
{{ $LOCATION_RCLONE_REPO := "bucket:restic-backup-12345" }}
{{ $LOCATION_RCLONE_CONFIG := "/home/deck/Desktop/.config/restic/config/rclone.conf" }}
{{ $LOCATION_RESTICPROFILE_LOCK := "/tmp/resticprofile-default.lock" }}
{{ $LOCATION_POSTGRES_DUMP := "/home/deck/Desktop/dumps" }}
{{ $LOCATION_PRIMARY_BACKUP_SOURCE := "/home/deck/Desktop/" }}

# Configs
{{ $CONFIG_CURRENT_TIME := .Now.Format "20060102T150405" }}
{{ $CONFIG_RESTIC_PASSWORD := "/home/deck/Desktop/.config/restic/config/password.txt" }}
{{ $CONFIG_RESTIC_EXCLUDE := "/home/deck/Desktop/.config/restic/excludes.txt" }}

global:
  default-command: snapshots                      # Run 'snapshots' when no command is specified
  initialize: false                               # Do not initialize a repository if none exists
  priority: low                                   # Use priority class on Windows and "nice" on Unixes
  min-memory: 100                                 # Minimum required RAM for Resticprofile to start
  restic-lock-retry-after: 5m                     # Retry failed restic command acquisition every 5 minutes
  restic-stale-lock-age: 10h                      # Unlock stale lock if age exceeds 10 hours
  restic-binary: '{{ $LOCATION_RESTIC_BINARY }}'  # Location of the Restic binary

default:
  lock: '{{ $LOCATION_RESTICPROFILE_LOCK }}'      # Local lockfile to prevent concurrent profile runs
  force-inactive-lock: true                       # Detect and remove stale locks
  initialize: true                                # Initialize repository if it doesn't exist
  repository: '{{ $LOCATION_RESTIC_REPO }}'       # Path to Restic repository
  password-file: '{{ $CONFIG_RESTIC_PASSWORD }}'  # File containing repository password
  status-file: '{{ $LOCATION_RESTIC_STATUS }}/{{ $CONFIG_CURRENT_TIME }}-restic-status.json'  # Output status file
  compression: 'max'                              # Maximum compression level
  run-after-fail:                                 # Block syncing if there was a failure. TODO: Add an email
    - 'echo "The command ${PROFILE_COMMAND} has failed in ${PROFILE_NAME}. Please check the logs." > {{ $LOCATION_RESTIC_BLOCKED_FILE }}'

  backup:
    run-before:                                   # Bring down Docker before backup
      - 'systemctl stop docker.socket'
      - 'systemctl stop docker'
    run-finally:
      - 'grep --invert-match -E "^unchanged|\(0 B added, 0 B stored\)|\(0 B added\)" {{ tempFile "backup.log" }} > {{ $LOCATION_RESTIC_LOG }}/{{ $CONFIG_CURRENT_TIME }}-restic-backup.log'  # Copy log file, stripping out any unchanced files
      - 'systemctl start docker'                  # Bring Docker back online after backup
    one-file-system: false                        # Exclude other file systems
    no-error-on-warning: true                     # Don't consider warnings as backup failures
    source:                                       # Directories to back up
      - '{{ $LOCATION_PRIMARY_BACKUP_SOURCE }}'
    exclude-file: '{{ $CONFIG_RESTIC_EXCLUDE }}'  # File containing exclude patterns
    exclude-caches: true                          # Exclude cache files
    schedule: '{{ $SCHEDULE_RESTIC_BACKUP }}'     # Backup schedule
    schedule-permission: system                   # Schedule permission
    schedule-lock-wait: 10m                       # Wait time for the lock during schedule
    schedule-log: '{{ tempFile "backup.log" }}'   # Log file to /tmp. This contains all information, including unchanged files which we do not care about
    verbose: 2                                    # Log details about processed files

  check:
    schedule: '{{ $SCHEDULE_RESTIC_CHECK }}'      # Verification schedule
    schedule-permission: system                   # Schedule permission
    schedule-lock-wait: 10m                       # Wait time for the lock during schedule
    schedule-log: '{{ $LOCATION_RESTIC_LOG }}/{{ $CONFIG_CURRENT_TIME }}-restic-check.log'  # Log file
    read-data: true                               # Verify data during check

  prune:
    dry-run: true                                 # Only prune if safe to do so, change manually
    repack-uncompressed: true                     # Repack all uncompressed data

  forget:
    dry-run: true                                 # Only forget if safe to do so, change manually

  rewrite:
    dry-run: true                                 # Only rewrite if safe to do so, change manually
    forget: true                                  # Remove original snapshots after creating new ones
    exclude-file: '{{ $CONFIG_RESTIC_EXCLUDE }}'  # File containing exclude patterns

  mount:
    allow-other: true                             # Allow other users to access the mount point

  rebuild-index:
    read-all-packs: true                          # Read all pack files to generate new index from scratch

# The following shell profiles are simply to run other shell scripts at a scheduled time
# We do not actually run the primary Restic commands listed, as we exit the process early

shell-postgres:                                   # Profile to run shell scripts only. We exit the current process before Restic can run.
  backup:
    schedule: '{{ $SCHEDULE_POSTGRES_BACKUP }}'   # Postgres backup schedule
    schedule-permission: system                   # Schedule permission
    schedule-lock-mode: ignore                    # Ignore locks, if any
    schedule-log: '{{ $LOCATION_RESTIC_LOG }}/{{ $CONFIG_CURRENT_TIME }}-postgres-backup.log'  # Log file
    dry-run: true                                 # Don't write data
    run-before:                                   # Dump postgres databases
      - 'chmod 777 /var/run/docker.sock'
      - 'docker exec -t immich-postgres pg_dumpall -c -U postgres | gzip > "{{ $LOCATION_POSTGRES_DUMP }}/immich-dump-{{ $CONFIG_CURRENT_TIME }}.sql.gz" && echo "Dumped Immich database: {{ $LOCATION_POSTGRES_DUMP }}/immich-dump-{{ $CONFIG_CURRENT_TIME }}.sql.gz"'
      - 'docker exec -t joplin-postgres pg_dumpall -c -U joplin | gzip > "{{ $LOCATION_POSTGRES_DUMP }}/joplin-dump-{{ $CONFIG_CURRENT_TIME }}.sql.gz" && echo "Dumped Joplin database: {{ $LOCATION_POSTGRES_DUMP }}/joplin-dump-{{ $CONFIG_CURRENT_TIME }}.sql.gz"'
      - 'kill $$'

shell-sync:
  backup:
    schedule: '{{ $SCHEDULE_SYNC_BACKUP }}'       # Sync backup schedule
    schedule-permission: system                   # Schedule permission
    schedule-lock-mode: ignore                    # Ignore locks, if any
    schedule-log: '{{ $LOCATION_RESTIC_LOG }}/{{ $CONFIG_CURRENT_TIME }}-rsync-backup.log'  # Log file
    dry-run: true                                 # Don't write data
    run-before:                                   # Sync the Restic repo, after checking if the repository is in good health
      - 'if [ -f "{{ $LOCATION_RESTIC_BLOCKED_FILE }}" ]; then echo "There has been a problem with the Restic repository, please check the logs. If everything is okay, delete the BLOCKED file." && kill $$; fi'
      - '{{ $LOCATION_RCLONE_BINARY }} -v sync {{ $LOCATION_RESTIC_REPO }} {{ $LOCATION_RCLONE_REPO }} --config={{ $LOCATION_RCLONE_CONFIG }} --b2-hard-delete'
      - '{{ $LOCATION_RCLONE_BINARY }} cleanup {{ $LOCATION_RESTIC_REPO }} --config={{ $LOCATION_RCLONE_CONFIG }}'
      - 'kill $$'

Resticprofile doesn't let me run other shell commands on a schedule, and because I wanted everything in a single configuration, I just created two new profiles which call the backup command. I then made the shell commands run before Restic, and then finally killed the instance before it got to actually run, which effectively does what I needed.

[-] vd1n@sh.itjust.works 1 points 1 year ago

Tears... Natural, salty, wet tears...

[-] kennyboy55@feddit.nl 1 points 1 year ago

I have an unraid server which hosts an docker image of Duplicacy. It is paid though for the web interface. And it backs up to Backblaze B2. I have roughly 175GB backed up, for which I pay $0.87 a month.

[-] lal309@lemmy.world 0 points 1 year ago

Do you have other clients backing up to your unraid? I’m looking for a complete solution to backing up end user workstations (windows, Mac and Linux) to my unraid server then backing up my unraid server to something like wasabi, Amazon, backblaze, etc. Preferably a single solution.

[-] kennyboy55@feddit.nl 1 points 1 year ago

Yes, I have another server automatically rsyncing important config files to a nfs share. And my pc has a samba share where I manually backup files to.

[-] Rakn@discuss.tchncs.de 1 points 1 year ago* (last edited 1 year ago)

Look into Veeam. The free version should be enough for this workflow.

[-] hollyberries@lemmy.blahaj.zone 1 points 1 year ago

I use Duplicati connected to Storj with data volumes that incrementally get backed up once per month. My files don't change very often, so monthly is a good balance. Not counting my Jellyfin library, those backups are around 1 TB. With the Jellyfin library, almost 15 TB.

Earlier this year, I recovered from a 100% data loss scenario, as I didn't (and still don't) have space for physical backups. I have a 25 TB allowance, so my actual cost was €0. If I had to pay, it would have been under €1.

[-] gamer@lemm.ee 1 points 1 year ago

That looks like a cool setup, but I would never trust important data to some crypto shit (Storj) no matter what kind of track record they have.

[-] hollyberries@lemmy.blahaj.zone 3 points 1 year ago

That's fair. I'm 100% onboard the decentralisation train, and do my hardest to practice what I preach. In the event that the service does go bust, I can make a backup on a different S3 compatible service immediately as long as my working copy is intact. The likelihood of the backup service AND the working copy dying at the exact same time would be my cue to take up knitting.

[-] alligatorSoup@feddit.uk 1 points 1 year ago

Do you mean 25TB as the storj site says 25gb? Did some promotion give you that much free?

[-] hollyberries@lemmy.blahaj.zone 0 points 1 year ago* (last edited 1 year ago)

Definitely 25 TB. I've used the service for a long time, since before they accepted credit cards. I attached my credit card one day and got a bump to 25 TB. Since that happened, I pay basically nothing and my account is still 100% storj token funded.

Edit: I dug up screenshots I sent someone recently

[-] Deebster@lemmyrs.org 1 points 1 year ago* (last edited 1 year ago)

my account is still 100% storj token funded

That seems to be the key bit, since everyone can use up to 25TB (if they can pay for it). Are you also hosting a node to earn ~~credits~~ tokens?

[-] GlitzyArmrest@lemmy.world 1 points 1 year ago

Regardless of service, if you don't test your backups, you have none.

[-] witten@lemmy.world 1 points 1 year ago

Ehhh I would say then you have probabilistic backups. There's some percent chance they're okay, and some percent chance they're useless. (And maybe some percent chance they're in between those extremes.) With the odds probably not in your favor. 😄

[-] Zikeji@programming.dev 2 points 1 year ago
[-] witten@lemmy.world 1 points 1 year ago
[-] pacjo@lemmy.world 1 points 1 year ago

Not so much about testing, but one time I really needed to get to my backups I lost password to the repository (I'm using restic). Luckily a copy of it was stored in bitwarden, but until I remembered it, were perhaps one of the worst moments.

Needless to say, please test backups and store secrets in more then one place.

[-] kalleboo@lemmy.world 1 points 1 year ago

Backblaze B2 for automatic syncing of all the little files

Glacier for long term archiving of old big files that never change

[-] cctl01@feddit.nl 1 points 1 year ago

Duplicati to Backblaze B2 for the important stuff. For as far as the media library goes, no backup just local raid setup...

[-] johntash@eviltoast.org 1 points 1 year ago

rsync.net is great if you need something simple and cheap. Backblaze B2 is also decent, but does have the typical download and API usage cost.

[-] Qu4ndo@discuss.tchncs.de 1 points 1 year ago

Borgbase with Borgmatic (Borg) as the Software. As far as I know the whole Borgbase Service is from a Homelab guy (with our needs in mind).

Also 3-2-1 rule!

[-] quantum_mechanic@lemmy.world 0 points 1 year ago* (last edited 1 year ago)

My truenas backs up to B2 Backblaze. Set it up years ago and haven't touched it since.

[-] atzanteol@sh.itjust.works 1 points 1 year ago

(you should test your backups)

You may have, but this is a friendly reminder just in case.

[-] quantum_mechanic@lemmy.world 1 points 1 year ago

Yeah I have. I work in tech, so I know better :)

[-] beerclue@lemmy.world 0 points 1 year ago

I used to have everything backed up to a 2TB USB drive. Which I accidentally dropped down the stairs. I lost thousands of family photos and documents. That changed my backup perspective.

I now have a Synology NAS, with 12TB in a RAID5 array (for a bit of disk redundancy). All my home devices, Proxmox servers etc back up here. The NAS also holds a few TB of media. Attached to it I have a USB hard drive (also 12TB). The NAS gets fully backed up to the USB drive nightly.

I also have a remote Raspberry Pi with a smaller USB drive (4TB) attached to it at my brother's house (in another country), where I backup most of the contents of my home NAS. I don't back up the media, just the important stuff. I might have to upgrade to a larger drive...

[-] theamigan@lemmy.dynatron.me 1 points 1 year ago

I used to have everything backed up to a 2TB USB drive. Which I accidentally dropped down the stairs. I lost thousands of family photos and documents. That changed my backup perspective.

If it's the only copy, it's not a backup. It's the master.

[-] shadowbert@kbin.social 0 points 1 year ago

Duplicati, to a friend's home server who lives in another town.

[-] GlitzyArmrest@lemmy.world 1 points 1 year ago

I hate to ask the scary question, but have you tried to restore your backups before? I used Duplicati and discovered that none of my backups were usable and ended up switching to Duplicacy.

[-] shadowbert@kbin.social 1 points 1 year ago

An important question though.

I have, when I first set it up, and again once when I needed to.

this post was submitted on 22 Jul 2023
12 points (100.0% liked)

Selfhosted

39251 readers
202 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS