Home server

[JPortici]

I'm not sure if it's the right section, but in the following months i would like to build myself a home server. The thing would have to accomplish several tasks:
- Private Git server
- Home Automation? (Maybe, but i'm going to need that for a project anyway..)
- NAS, maybe media server?
- Backups (of NAS/Git)
- Other, for experiments if i don't want to run virtual machines on my PC

The first question i want to ask: Could it make sense to have a single PC with a barebones linux and a virtual machine for every task, instead of several smaller machines.. or instead of running everything on one OS, which i don't really want to do.
Second question: It would probably be running 24/7, so what would i be looking for? lower power intel mini pc? a more powerful SBC? can ARM SBC create and use virtual machines? Though i probably want a separate hard disk for every image, or at least separate hard disk for backups, and that may rule out most SBCs in a sensitive price range

[DiTBho]

I'm the wrong person to suggest other things, because... well, I do very weird things with weird hw (I am writing on GNU/Linux SH3 laptop, at the moment), but I can suggest one thing (and, being x86, it's strange that I suggest it myself)

have a look at Zimaboards, see here 

that stuff supports containers, XEN, qemu ... and it looks perfect for your needs.

[bitwelder]

At least for Git server, home automation and other tasks that don't require extensive storage I'd suggest to go with something like a (2nd hand) Lenovo ThinkCentre m920q (or similar m900q model), in Tiny form-factor, i.e. it's a small box less than 20x20x4 cm, with a recent Intel processor.
It could either run some hypervisor like Proxmox, so to have totally independent virtual machines, or some standard Linux distro and use e.g. docker to just containerize your appplications.
HP has also a line of tiny PCs, I think it's named Elite Mini 800.

Depending on the storage needs for the backup/NAS functions, perhaps it could be used also for that  (it's a small box, so internally there is not more space than a couple of 2.5" disk and perhaps an M.2 module)
 

[madires]

The first question i want to ask: Could it make sense to have a single PC with a barebones linux and a virtual machine for every task, instead of several smaller machines.. or instead of running everything on one OS, which i don't really want to do.

Yes, but it depends on your requirements. Another alternative could be Docker.

Second question: It would probably be running 24/7, so what would i be looking for? lower power intel mini pc? a more powerful SBC? can ARM SBC create and use virtual machines? Though i probably want a separate hard disk for every image, or at least separate hard disk for backups, and that may rule out most SBCs in a sensitive price range

The most important thing for you is to figure out what exactly you need. Your short list doesn't tell much. Maybe a NAS supporting VMs could be sufficient.

[JPortici]

Maybe a NAS supporting VMs could be sufficient.

does such a thing even exist? (rethorical question i guess) I always thought them/treated them as dumb boxes that expose an HDD to the network.

[madires]

VMs are supported for many years by the two main SOHO vendors:
- https://www.synology.com/en-global/dsm/overview/virtualization
- https://www.qnap.com/en/software/virtualization-station

[dobsonr741]

Get TrueNAS, earlier called as FreeNAS. https://www.truenas.com/
I’m using it for many years, to back up to it. Just had a Mac SSD crashing, TrueNAS saved the day.

One caveat: max out the RAM on the mobo. 32GB at least, better to have 64GB. 8GB boots, but clearly not recommended.

You can have docker instances on it, plus endless number of prepackaged plugins.

[Halcyon]

A hypervisor like Proxmox or XCP-ng is a great idea for running various different systems or services on your network. However, whilst it'll work just fine, I generally avoid running a NAS (like TrueNAS) and routers (pfSense, OPNsense) as a virtual machine.

Firstly, if you're dealing with ZFS, the OS likes to be in direct control of the disk hardware. The risk is that data corruption that occurs outside the guest OS's control may not be detected, defeating the purpose of ZFS's data resilience.

As for the router, if for whatever reason there is an issue with your hypervisor or the router system, you lose internet connectivity that you might otherwise need to troubleshoot any issues.

It's mostly about not putting all your eggs in one basket.

Maybe a NAS supporting VMs could be sufficient.

does such a thing even exist? (rethorical question i guess) I always thought them/treated them as dumb boxes that expose an HDD to the network.

Yep, you can run virtual machines inside TrueNAS. https://www.truenas.com/docs/core/coretutorials/jailspluginsvms/virtualmachines/
If your limited to just one machine, this is the way I'd do it. TrueNAS as the underlying OS with your other services running virtually within it.

[JPortici]

A hypervisor like Proxmox or XCP-ng is a great idea for running various different systems or services on your network. However, whilst it'll work just fine, I generally avoid running a NAS (like TrueNAS) and routers (pfSense, OPNsense) as a virtual machine.

Firstly, if you're dealing with ZFS, the OS likes to be in direct control of the disk hardware. The risk is that data corruption that occurs outside the guest OS's control may not be detected, defeating the purpose of ZFS's data resilience.

As for the router, if for whatever reason there is an issue with your hypervisor or the router system, you lose internet connectivity that you might otherwise need to troubleshoot any issues.

It's mostly about not putting all your eggs in one basket.

Maybe a NAS supporting VMs could be sufficient.

does such a thing even exist? (rethorical question i guess) I always thought them/treated them as dumb boxes that expose an HDD to the network.

Yep, you can run virtual machines inside TrueNAS. https://www.truenas.com/docs/core/coretutorials/jailspluginsvms/virtualmachines/
If your limited to just one machine, this is the way I'd do it. TrueNAS as the underlying OS with your other services running virtually within it.

Yesterday after work i was talking to ourIT guy and he actually told me most of this.

Let's say i get a NAS in which i can install TrueNAS/FreeNAS, and the NAS have two hard disks.
I could dedicate one HDD to NAS functionality, and the other to the various VMs, every VM in its own partition.
Would that solve the problem of potentially mangling the data? (which besides i didn't know about, thanks. Also filesystems are a mystery to me at the present moment. I sometimes read threads about them but most of it is currently lost)

Otherwise i could install linux (or i actually get a Mini PC and install linux) + Proxmox - or another hypervisor - and then to one HDD i run the VM with NAS functionality, on the other hard disk i run the other VMs. That would surely solve the problems about the NAS mangling the data, correct? and then achieve more separation in the sense that i can log in to the PC (i attach a monitor and a keyboard) and then stop/restart whatever VM is causing network problems, if any, without having to restart the whole thing

[DiTBho]

A hypervisor like Proxmox or XCP-ng is a great idea for running various different systems or services on your network. However, whilst it'll work just fine, I generally avoid running a NAS (like TrueNAS) and routers (pfSense, OPNsense) as a virtual machine.

Firstly, if you're dealing with ZFS, the OS likes to be in direct control of the disk hardware. The risk is that data corruption that occurs outside the guest OS's control may not be detected, defeating the purpose of ZFS's data resilience.

Yup. The same thing I think 

[DiTBho]

filesystems

For the final user's point of view, there isn't much to say about file systems
There are

• those from the DOS/Windows world { FAT, FAT32, ExFat, NTFS, ... }
• those from the BSD world and/or strange OSes (like BeOS/Haiku)
• those from the commercial UNIX world
• those from the Linux world { EXT2, EXT3, EXT4, XFS, BTRFS, ... BCacheFS, ... }
• those for Networking { NFS, SAMBA, ... }

From the user point of view, among the Linux ones there are 4 categories:

• very simple, without journaling or other features, e.g. { MINIXFS, EXT2, ... }
  designed only to make life easier for simple bootloaders like LILO or old versions of GRUB, or things like that
• traditional with journaling, but without any metadata control. , e.g. { EXT3, EXT4, XFS, ... }
  they only guarantee that an IO operation was recorded (into the "journal", an internal data-structure, securelay saved into a special disk area) as successful or not, they do not offer any guarantee of data consistency. These file systems require active S.M.A.R.T. monitoring to understand if a disk is starting to behave erratically, causing silent data corruption. It's not the filesystem that reports it, when you make a copy of a file via the traditional "/bin/cp" program, no one checks that the copy has exactly the same checksum as the original.
  They don't even provide any RAID functionality. You can use softRAID in kernel space, but that's a separate thing
• advanced file systems, offering both data consistency control and RAID functionality, as well as  "snapshots" via copy-on-write
  The most common examples are { ZFS, BTRFS(1), ... }
• Experimental filesystems, e.g. { BCacheFS, ... }

(1) BTRFS is rock solid as long as you don't use its RAID features.
Still buggy.

[Monkeh]

(1) BTRFS is rock solid as long as you don't use its RAID features.

So long as you have a completely reliable, error-correcting medium under it.

[Halcyon]

Let's say i get a NAS in which i can install TrueNAS/FreeNAS, and the NAS have two hard disks.
I could dedicate one HDD to NAS functionality, and the other to the various VMs, every VM in its own partition.
Would that solve the problem of potentially mangling the data? (which besides i didn't know about, thanks. Also filesystems are a mystery to me at the present moment. I sometimes read threads about them but most of it is currently lost)

Otherwise i could install linux (or i actually get a Mini PC and install linux) + Proxmox - or another hypervisor - and then to one HDD i run the VM with NAS functionality, on the other hard disk i run the other VMs. That would surely solve the problems about the NAS mangling the data, correct? and then achieve more separation in the sense that i can log in to the PC (i attach a monitor and a keyboard) and then stop/restart whatever VM is causing network problems, if any, without having to restart the whole thing

Don't think of it in the way of allocating various drives to various purposes, that's what ZFS pools/datasets are for. Your NAS should be configured to use all available drives, then create a dataset specific to your VMs or jails. The data will sit on all the drives (along with all your other datasets) but for all intents and purposes "on its own" with it's own set of permissions (and shares if you like).

Datasets are a bit like partitions on a drive.

[nightfire]

From my experience as a sysadmin I see two possible path you could pursue:

- Buying some Nas system from the "usual suspects" like Qnap or Synology, and go with their ecosystem. Regarding virtualisation, you probably would want to use SSDs inside for good I/O.  You should use a 4-slot chassis or more slots, so that you can grow, and maybe use for data storage cheaper HDDs that for the I/O-sensitive part like hosting VMs.
- You build your own NAS from spare parts. Nowadays it is easy to get old computers (hint: some refurbishers buy old corporate gear and resell it for cheap with some warranty) and put some linux distro on it, or use some ready-made system like TrueNAS. While you go this road, you should clearly define which amount of data you will have to handle, and which availability it should have.

Regarding energy consumption: For PC systems, that are small, I liked the mini PC types like the Prodesk or Elitedesk Mini from HP- they have basically to form factor of a NUC and are powered by an external notebook power supply. You can get easily 16 GB main memory in them, and swap out the internal SSD to some bigger SATA model.
If the change rate of your data is not high, backup to external USB HDD should be fine.
Alternatively, you check the costs of new hardware versus the energy costs- for getting first experiences about building some NAS it ca be overall cheaper to re-use old PC parts instead buying new stuff, if the old parts are still reliable. So you could go for an old PC, put 2 mirrored SSDs in for the main system, and 2 HDDs for data storage. Some mini-ATX chassis should be sufficient to do this, and even with an old quad-core CPU from the Haswell times there should be enough power to drive that.

[DiTBho]

(1) BTRFS is rock solid as long as you don't use its RAID features.

So long as you have a completely reliable, error-correcting medium under it.

it's true that, LOL 

[DiTBho]

Synology

I also often recommend it
Haven't you seen that they shoot at 900Mbyte/sec on 10Gbps ethernet ink?
there are even people who do video editing directly on the NAS, via NFS 

[SiliconWizard]

Synology

I also often recommend it
Haven't you seen that they shoot at 900Mbyte/sec on 10Gbps ethernet ink?
there are even people who do video editing directly on the NAS, via NFS 

Well, at this throughput, no wonder. People were (and some still are, for cost reasons) doing video editing on large HDDs with throughput in the 100MB/s-200MB/s for the best ones. 900MB/s is several times faster.
I don't know if we're getting spoiled, or something.

[DiTBho]

I don't know if we're getting spoiled, or something.

they are spoiling us, even 10 Gbit/sec internet connection is something like that.
Youtubers 

[SiliconWizard]

I don't know if we're getting spoiled, or something.

they are spoiling us, even 10 Gbit/sec internet connection is something like that.
Youtubers 

I don't know how many have a 10Gb/s internet access. Not me.

[nightfire]

Synology

I also often recommend it
Haven't you seen that they shoot at 900Mbyte/sec on 10Gbps ethernet ink?
there are even people who do video editing directly on the NAS, via NFS 

I have such a Synology storage at work, equipped with 8 HDDs, and 10 Gbit/s option. Nice stuff, but sustainedd data rate is rather somewhat in the 600-700 Mbyte/s range, running RAID6. You can see where RAM buffers a lot, but it gets every few moments a performance drop probably due to syncing data within the RAID or flushing caches.
Still, for the typical usage pattern of a video edit system it is quite ok, if the I/O of your physical media allows for it. Also it allows to put all your media in a big NAS and eliminating the need for further SSDs in your editing rig.

[Berni]

I personally run everything on single x86 PC running Unraid for the NAS and then have VMs or Dockers inside of that to run everything else.

Most diy NAS software solutions will give you the option to run VMs on it (usually using KVM) and you can do plenty advanced stuff with that. You can even passtrough hardware into the VM, letting you have a VM performant enough to play modern AAA games on it when using GPU passtrough. For the filesystem most NAS solutions have a preferred filesystem so just use that.

Having it all in one box like that makes managing it all very easy because you just log into the web UI and you can manage everything. See total resource usage, start/kill VMs, VNC into each VM, install stuff, check logs..etc.

Tho for something like home automation it is probably a good idea to run that on a separate box. Such a task doesn't need much computing horsepower so it could run on a little NUC PC or a old laptop. That way if the big server goes down or is having maintenance done to it, the home automation keeps running fine. Similar reliability reason why it is a good idea to use a separate machine as a router, you want the network to stay up (Tho in my opinion a off the shelf router is plenty good as long as you buy something better than cheap home routers).