From 289769a60114d64b05c9f1dde3877ae4b6fa1878 Mon Sep 17 00:00:00 2001 From: Nicole Rappe Date: Sat, 5 Sep 2026 14:08:43 -0600 Subject: [PATCH] Restructured Documentation --- blog/index.md | 22 +- ...es Causing Notable CPU Performance Loss.md | 15 +- ...2025 Learning to Leverage Gitea Runners.md | 86 +-- ...12-15-2024 Implementing the Blog Plugin.md | 8 +- .../12-15-2024 OpenStack Frustrations.md | 10 +- .../Asset Management/Homebox.md | 12 +- .../Asset Management/Snipe IT.md | 13 +- .../Communication/Niltalk.md} | 7 +- .../Rocketchat/Deploy Rocket.Chat.md} | 20 +- .../Dashboards/Dashy.md} | 15 +- .../Dashboards}/Homepage Docker.md | 13 +- .../Configuring ACME Letsencrypt Bot.md | 79 +++ .../Integrate PMG with Mailcow.md} | 492 +++------------- .../cPanel/Create a cPanel Email Server.md | 17 + .../Email}/iRedMail/Deploy iRedMail.md | 54 +- .../email => Applications/Email}/mailcow.md | 34 +- .../Collabora Code Server.md | 17 +- .../Files and Collaboration}/Nextcloud AIO.md | 19 +- .../Files and Collaboration/Nextcloud.md} | 9 +- .../Files and Collaboration/OnlyOffice EE.md | 76 +++ .../Files and Collaboration}/Pyload.md | 14 +- .../Files and Collaboration}/Stirling PDF.md | 11 +- .../Files and Collaboration/Trilium.md} | 12 +- .../DFS Namespaces with Replication.md | 134 +++++ .../Files and Collaboration/Wordpress.md} | 7 +- .../Ark Survival Ascended.md | 19 +- .../Gaming and Media}/Emulatorjs.md | 11 +- .../Gaming and Media/Pterodactyl.md} | 16 +- .../Gaming and Media/Valheim.md} | 11 +- .../Home Automation}/Frigate.md | 11 +- .../Home Automation}/HomeAssistant.md | 10 +- .../Monitoring/Gatus.md} | 17 +- .../Monitoring/Ntfy.md} | 7 +- .../Monitoring}/Speedtest Tracker.md | 18 +- .../Monitoring/UptimeKuma.md} | 7 +- .../Utilities}/Changedetection.md | 17 +- .../Applications/Utilities/Cyberchef.md | 40 ++ .../Applications/Utilities/IT Tools.md | 38 ++ .../Utilities}/Searx.md | 11 +- deployments/Applications/index.md | 24 + .../kopia.md => Backup and Recovery/Kopia.md} | 9 +- deployments/Backup and Recovery/index.md | 16 + .../Docker/Create the Docker Network.md | 11 +- .../Docker/Deploy Portainer.md | 24 +- .../Containers/Docker/Git Repo Updater.md | 69 +++ .../k8s.md => Containers/Kubernetes/K8s.md} | 15 +- .../Kubernetes}/Rancher RKE2.md | 63 +- deployments/Containers/index.md | 17 + .../Active Directory/Certificate Services.md | 165 ++---- .../Authelia.md} | 9 +- .../Authentik.md} | 19 +- .../Keycloak/Deploy Keycloak.md} | 26 +- .../Password Pusher.md | 11 +- .../Privacyidea.md} | 22 +- .../Vaultwarden.md | 11 +- .../Identity and Certificates/index.md | 22 + .../Networking and Access/DNS/AdGuard Home.md | 38 ++ .../DNS/Pi Hole.md | 7 +- .../NetBird/Deploy NetBird on Rocky Linux.md | 7 +- .../Remote Access/Apache Guacamole.md | 20 +- .../Remote Access/Firefox.md | 11 +- ...tall XFCE and RustDesk on Fedora Server.md | 13 +- .../Remote Access/Install XRDP on Ubuntu.md | 9 +- .../Reverse Proxies/Nginx.md} | 7 +- .../Reverse Proxies/Traefik.md} | 21 +- .../Configure a Site-to-Site IPsec VPN.md | 23 +- .../Deploy UniFi Controller with Docker.md | 8 +- .../Deploy UniFi Network Server on Ubuntu.md | 13 +- deployments/Networking and Access/index.md | 21 + .../Guests/Install the QEMU Guest Agent.md | 9 +- .../Deploy Failover Cluster Node.md | 97 ++-- .../OpenStack/Ansible OpenStack.md | 27 +- .../OpenStack/Canonical OpenStack.md | 46 +- ...Configuring ISCSI Based Cluster Storage.md | 24 +- .../Create an Ubuntu Cloud-Init Template.md} | 21 +- .../Proxmox/Deploy Proxmox VE.md} | 52 +- .../Proxmox}/ZFS over ISCSI.md | 30 +- .../Rancher Harvester/Harvester.md | 64 +++ .../Virtualization and Storage/index.md | 20 + .../Fedora/Set Up the Fedora Workstation.md | 11 +- deployments/Windows and Linux/index.md | 16 + .../awx/Deployment => AWX}/AWX Operator.md | 60 +- .../awx/Deployment => AWX}/AWX in Minikube.md | 83 +-- .../{ansible/awx/awx.png => AWX/AWX.png} | Bin .../Documentation/Docusaurus.md} | 7 +- .../Documentation}/Material MkDocs.md | 10 +- .../Documentation/Zensical.md} | 146 +---- .../FOG Project/Configure Pi-Hole PXE Boot.md | 11 +- .../Configure Windows Server PXE Boot.md | 15 +- .../FOG Project/Deploy FOG Project.md | 184 +++--- .../gitea.md => automation/Gitea/Gitea.md} | 12 +- .../deployment => Puppet}/Puppet Bolt.md | 47 +- .../deployment/puppet.md => Puppet/Puppet.md} | 89 +-- .../Remote Management/TacticalRMM.md} | 13 +- .../Tools}/Activepieces.md | 13 +- deployments/automation/Tools/Node Red.md | 41 ++ .../Tools}/Semaphore UI.md | 12 +- deployments/automation/index.md | 38 +- deployments/index.md | 27 +- .../Custom Containers/Git Repo Updater.md | 114 ---- deployments/platforms/index.md | 37 -- .../Rancher Harvester/Harvester.md | 57 -- .../services/Automation Tools/Node Red.md | 36 -- deployments/services/DNS/AdGuard Home.md | 37 -- .../DFS Namespaces with Replication.md | 542 ------------------ .../Security and Utility/Cyberchef.md | 35 -- .../services/Security and Utility/IT Tools.md | 33 -- .../keycloak/oauth2/Gitea OAuth2.md | 20 - .../keycloak/oauth2/deployment.md | 10 - .../services/cPanel/Creating Email Server.md | 16 - .../Configuring ACME Letsencrypt Bot.md | 78 --- .../email/iRedMail/Quick Server Settings.md | 9 - deployments/services/index.md | 40 -- .../services/productivity/OnlyOffice EE.md | 70 --- .../Email/iRedMail Connection Settings.md | 18 + reference/Applications/Email/index.md | 31 + .../Rclone Command Reference.md | 101 ++++ .../Robocopy Command Reference.md | 16 +- .../Files and Collaboration/index.md | 35 ++ reference/Applications/index.md | 23 + .../AWX/Credential Configuration Examples.md | 48 ++ .../AWX/Custom Kerberos WinRM Credential.md | 13 +- .../AWX/Inventory Structure and Variables.md | 8 +- .../AWX/Job Template Configuration.md | 11 +- .../Automation/AWX/Playbook Catalog.md | 35 +- .../AWX/Projects and Source Control.md | 21 + reference/Automation/AWX/index.md | 36 ++ .../Gitea Configuration Delivery.md | 23 + reference/Automation/index.md | 23 + .../Backup and Recovery/Veeam Concepts.md | 23 +- .../Veeam Retention Policy Example.md | 10 +- reference/Backup and Recovery/index.md | 23 + reference/Containers/index.md | 23 + .../LDAP Connection Settings.md | 8 +- .../Keycloak Integrations.md | 20 + reference/Identity and Certificates/index.md | 23 + .../Hardware/Cluster Node 01 Disk Layout.md} | 5 +- .../Hardware/Cluster Node 02 Disk Layout.md} | 5 +- .../Hardware/Cluster Node 03 Disk Layout.md} | 5 +- ...orage Node 01 TrueNAS Core Disk Layout.md} | 5 +- ...rage Node 02 TrueNAS Scale Disk Layout.md} | 5 +- .../Lab Map/Hardware/iLO License Reference.md | 10 +- .../Homelab Server Inventory.md | 37 +- .../172.16.16.0 Sophos Network.md | 10 +- .../192.168.5.0 Container Network.md | 11 +- .../Lab Map/Power/APC Battery Cell Wiring.md | 22 +- .../Lab Map/Power/UPS Power Distribution.md | 8 +- reference/Lab Map/index.md | 23 + .../DNS/Windows DNS Configuration Notes.md | 53 +- reference/Networking and Access/index.md | 23 + .../Proxmox/index.md | 39 ++ reference/Virtualization and Storage/index.md | 23 + .../Windows and Linux/Linux Time Commands.md | 10 +- reference/Windows and Linux/index.md | 23 + .../foundations/Choose a Document Template.md | 22 + .../foundations/Documentation Organization.md | 36 ++ .../foundations/Documentation Path Changes.md | 209 +++++++ .../Templates/Document Template.md | 38 -- reference/foundations/index.md | 48 +- reference/index.md | 28 +- reference/infrastructure/hardware/index.md | 38 -- reference/infrastructure/networking/index.md | 38 -- .../Connect to Exchange Online.md | 16 +- .../Set Mailbox Auto Reply.md | 13 +- .../Start Exchange Services.md | 27 + .../Report DFS Namespaces and Replication.md | 281 +++++++++ .../DFS/Report DFS Replication Backlog.md | 132 +++++ .../Directory Walker.md | 11 +- .../Files and Collaboration}/File Finder.md | 9 +- .../Report NTFS Permissions Across Shares.md} | 13 +- .../SMB/Report SMB Share Permissions.md} | 9 +- .../Upload Data to a Nextcloud Share.md} | 26 +- .../Update the ATM10 Minecraft Server.md} | 12 +- .../Blue Iris Server Watchdog.md} | 10 +- scripts/Applications/index.md | 19 + .../Gitea}/Git Repo Updater.md | 7 +- scripts/Automation/index.md | 16 + .../Bash/Fix Displaylink Issues on Linux.md | 12 - scripts/Bash/Mdadm Grow Array Size.md | 13 - scripts/Bash/Open Port Checker.md | 15 - scripts/Bash/Transfer Files with Netcat.md | 28 - ... Group Policy Updates Across the Domain.md | 17 + .../Reports}/Get Password Expiration.md | 9 +- .../Reports/Inactive Computers.md | 18 + .../Reports/Inactive Users.md | 19 + .../Microsoft 365}/Check Email Aliases.md | 13 +- .../Microsoft 365}/Connect to Azure AD.md | 16 +- scripts/Identity and Certificates/index.md | 17 + ...ge DNS Client Server Settings Remotely.md} | 13 +- .../Networking and Access/Check Open Ports.md | 18 + .../Correct DNS Server Priority.md} | 5 +- scripts/Networking and Access/index.md | 18 + .../Force GPUpdate Domain Wide.md | 11 - .../Reporting/Inactive Computers.md | 12 - .../Powershell/Reporting/Inactive Users.md | 13 - .../Restart Exchange Services.md | 20 - .../Collapse Differencing Disk Chains.md | 28 +- .../Hyper-V}/Delete Locked VHDX File.md | 7 +- .../Force Reboot Remote Cluster Node.md | 10 +- .../Failover Cluster/Replication Bumper.md | 13 +- .../Linux/Grow an mdadm Array.md | 20 + .../Proxmox}/Deeplab Rollback Script.md | 12 +- scripts/Virtualization and Storage/index.md | 18 + .../Clean Up Inactive User Profiles.md} | 8 +- .../Repair Windows Update Components.md} | 7 +- ...he RMM Agent Service Across the Domain.md} | 9 +- .../Upgrade Windows 11 from a UNC Path.md} | 13 +- scripts/Windows and Linux/index.md | 16 + scripts/index.md | 38 +- .../Configure Autotask Link Replacement.md | 9 +- .../Manage DAG Database Copies.md | 25 +- ...Perform Exchange SE DAG Rolling Updates.md | 13 +- .../Prepare for Cumulative Updates.md | 7 +- ...d Mail Delivery Between PMG and Mailcow.md | 329 +++++++++++ .../Inspect the Outgoing SMTP Queue.md | 5 +- .../Configure and Recover Rclone Bisync.md | 89 +-- ... an Inconsistent DFS Management Console.md | 41 ++ .../Access Another User OneDrive Data.md | 14 +- .../Transfer Files with Netcat.md | 34 ++ .../Connect Tuya Smart Lights.md | 44 +- workflows/Applications/index.md | 19 + .../AWX/AWX Kerberos Implementation.md | 28 +- .../AWX/Connect AWX to Gitea.md | 8 +- ...air Upgrades Beyond AWX Operator 2.10.0.md | 13 +- ...iver Configuration with a Docker Runner.md | 90 +++ ...nsical Documentation with a Host Runner.md | 147 +++++ workflows/Automation/index.md | 17 + .../Veeam}/Backup Agent Takeover.md | 14 +- ... Validate Certificates of Some Gateways.md | 8 +- .../Veeam}/Manually Pruning Backups.md | 9 +- .../Migrating VSPC Backup Repositories.md | 5 +- workflows/Backup and Recovery/index.md | 16 + .../Build and Publish a Container Image.md | 7 +- .../Docker/Create a Macvlan Subinterface.md | 13 +- ...ransfer Docker Containers Between Hosts.md | 24 +- .../Migrating Docker Compose YML to K8s.md | 22 +- workflows/Containers/index.md | 17 + ...Create a Desktop Shortcut to a UNC Path.md | 9 +- .../Active Directory/Restore Domain Trust.md | 5 +- .../Convert Certificates to PFX.md} | 15 +- .../Export Certificates for LDAPS Clients.md | 51 ++ ...d Maintain Certificate Revocation Lists.md | 77 +++ .../Keycloak/Connect Gitea to Keycloak.md | 27 + .../Keycloak/Connect Portainer to Keycloak.md | 9 +- .../Microsoft 365/Change MFA Settings.md | 8 +- .../Windows/Enable WinRM over HTTPS.md} | 7 +- workflows/Identity and Certificates/index.md | 20 + .../Linux/Change a Server IP Address.md | 11 +- .../Configure SSH Key Authentication.md | 8 +- .../Automatically Reset an IPsec Tunnel.md | 25 +- .../Sophos/Configure LAN Bridging.md | 9 +- .../Configure RDP Access over SSL VPN.md | 4 +- workflows/Networking and Access/index.md | 17 + .../Virtualization/Proxmox/Common Tasks.md | 12 - .../Rebuild Failover Cluster Replication.md | 18 +- .../Hyper-V}/Forcefully Stop GuestVM.md | 13 +- .../Hyper-V}/Kerberos Enabled VM Migration.md | 8 +- .../Linux/Expand a Linux Guest Filesystem.md} | 44 +- .../Expand an iSCSI-Backed ZFS Filesystem.md} | 9 +- .../Detect and Remove Orphaned VM Disks.md | 119 ++-- .../Manually Activate a Volume Group.md} | 6 +- .../Remove a Node from a Proxmox Cluster.md | 18 + ...ir Rocky Linux After a Veeam Migration.md} | 14 +- .../Repair iSCSI Connections After Reboot.md | 17 +- .../Upgrade Proxmox VE from 8 to 9.md} | 7 +- .../Replace a Drive in Storage Node 01.md | 10 +- workflows/Virtualization and Storage/index.md | 19 + ...estrict Monitors on Plasma Login Screen.md | 12 +- .../Install Flatpak Apps.md | 9 +- .../Fedora Workstation/Upgrading Versions.md | 7 +- .../Repair DisplayLink USB Authorization.md | 18 + .../Windows/Change Windows Edition.md | 41 +- .../Delete Windows Recovery Partition.md | 15 +- .../Windows/Uninstall Updates via DISM.md | 11 +- .../Windows/VSS/Delete Shadow Copies.md | 42 ++ .../Creating a Local Account on Win11.md | 8 +- workflows/Windows and Linux/index.md | 17 + workflows/index.md | 50 +- .../Windows/VSS/Delete Shadow Copies.md | 39 -- .../ansible/credentials/overview.md | 40 -- .../automation/ansible/projects/overview.md | 22 - 281 files changed, 5403 insertions(+), 3563 deletions(-) rename deployments/{services => Applications}/Asset Management/Homebox.md (71%) rename deployments/{services => Applications}/Asset Management/Snipe IT.md (84%) rename deployments/{services/communication/niltalk.md => Applications/Communication/Niltalk.md} (64%) rename deployments/{services/communication/rocketchat/deployment.md => Applications/Communication/Rocketchat/Deploy Rocket.Chat.md} (81%) rename deployments/{services/dashboards/dashy.md => Applications/Dashboards/Dashy.md} (70%) rename deployments/{services/dashboards => Applications/Dashboards}/Homepage Docker.md (71%) create mode 100644 deployments/Applications/Email/Microsoft Exchange/Configuring ACME Letsencrypt Bot.md rename deployments/{services/email/Proxmox Mail Gateway/Deploying PMG.md => Applications/Email/Proxmox Mail Gateway/Integrate PMG with Mailcow.md} (50%) create mode 100644 deployments/Applications/Email/cPanel/Create a cPanel Email Server.md rename deployments/{services/email => Applications/Email}/iRedMail/Deploy iRedMail.md (94%) rename deployments/{services/email => Applications/Email}/mailcow.md (78%) rename deployments/{services/productivity => Applications/Files and Collaboration}/Collabora Code Server.md (91%) rename deployments/{services/productivity => Applications/Files and Collaboration}/Nextcloud AIO.md (94%) rename deployments/{services/productivity/nextcloud.md => Applications/Files and Collaboration/Nextcloud.md} (78%) create mode 100644 deployments/Applications/Files and Collaboration/OnlyOffice EE.md rename deployments/{services/Media and Gaming => Applications/Files and Collaboration}/Pyload.md (78%) rename deployments/{services/productivity => Applications/Files and Collaboration}/Stirling PDF.md (59%) rename deployments/{services/productivity/trilium.md => Applications/Files and Collaboration/Trilium.md} (55%) create mode 100644 deployments/Applications/Files and Collaboration/Windows Server/DFS Namespaces with Replication.md rename deployments/{services/productivity/wordpress.md => Applications/Files and Collaboration/Wordpress.md} (65%) rename deployments/{services/gaming => Applications/Gaming and Media}/Ark Survival Ascended.md (90%) rename deployments/{services/Media and Gaming => Applications/Gaming and Media}/Emulatorjs.md (62%) rename deployments/{services/gaming/pterodactyl.md => Applications/Gaming and Media/Pterodactyl.md} (62%) rename deployments/{services/gaming/valheim.md => Applications/Gaming and Media/Valheim.md} (82%) rename deployments/{services/Home and IOT => Applications/Home Automation}/Frigate.md (71%) rename deployments/{services/Home and IOT => Applications/Home Automation}/HomeAssistant.md (51%) rename deployments/{services/monitoring/gatus.md => Applications/Monitoring/Gatus.md} (71%) rename deployments/{services/notifications/ntfy.md => Applications/Monitoring/Ntfy.md} (56%) rename deployments/{services/monitoring => Applications/Monitoring}/Speedtest Tracker.md (83%) rename deployments/{services/monitoring/uptimekuma.md => Applications/Monitoring/UptimeKuma.md} (64%) rename deployments/{services/Security and Utility => Applications/Utilities}/Changedetection.md (65%) create mode 100644 deployments/Applications/Utilities/Cyberchef.md create mode 100644 deployments/Applications/Utilities/IT Tools.md rename deployments/{services/Security and Utility => Applications/Utilities}/Searx.md (67%) create mode 100644 deployments/Applications/index.md rename deployments/{services/backup/kopia.md => Backup and Recovery/Kopia.md} (74%) create mode 100644 deployments/Backup and Recovery/index.md rename reference/infrastructure/networking/Docker Networking/Docker Networking.md => deployments/Containers/Docker/Create the Docker Network.md (52%) rename deployments/{platforms/containerization => Containers}/Docker/Deploy Portainer.md (81%) create mode 100644 deployments/Containers/Docker/Git Repo Updater.md rename deployments/{platforms/containerization/kubernetes/deployment/k8s.md => Containers/Kubernetes/K8s.md} (93%) rename deployments/{platforms/containerization/kubernetes/deployment => Containers/Kubernetes}/Rancher RKE2.md (95%) create mode 100644 deployments/Containers/index.md rename deployments/{services/authentication => Identity and Certificates}/Active Directory/Certificate Services.md (75%) rename deployments/{services/authentication/authelia.md => Identity and Certificates/Authelia.md} (52%) rename deployments/{services/authentication/authentik.md => Identity and Certificates/Authentik.md} (82%) rename deployments/{services/authentication/keycloak/deployment.md => Identity and Certificates/Keycloak/Deploy Keycloak.md} (91%) rename deployments/{services/Security and Utility => Identity and Certificates}/Password Pusher.md (77%) rename deployments/{services/authentication/privacyidea.md => Identity and Certificates/Privacyidea.md} (90%) rename deployments/{services/Security and Utility => Identity and Certificates}/Vaultwarden.md (73%) create mode 100644 deployments/Identity and Certificates/index.md create mode 100644 deployments/Networking and Access/DNS/AdGuard Home.md rename deployments/{services => Networking and Access}/DNS/Pi Hole.md (66%) rename reference/infrastructure/networking/vpn/netbird.md => deployments/Networking and Access/NetBird/Deploy NetBird on Rocky Linux.md (89%) rename deployments/{services => Networking and Access}/Remote Access/Apache Guacamole.md (87%) rename deployments/{services => Networking and Access}/Remote Access/Firefox.md (85%) rename workflows/operations/Linux/Fedora Workstation/Install DE into Fedora Server.md => deployments/Networking and Access/Remote Access/Install XFCE and RustDesk on Fedora Server.md (91%) rename scripts/Bash/Install XRDP.md => deployments/Networking and Access/Remote Access/Install XRDP on Ubuntu.md (74%) rename deployments/{services/edge/nginx.md => Networking and Access/Reverse Proxies/Nginx.md} (54%) rename deployments/{services/edge/traefik.md => Networking and Access/Reverse Proxies/Traefik.md} (83%) rename reference/infrastructure/networking/Firewall and Routing/Sophos/VPN/Site to Site VPNs/IPSEC/Tunnel Creation.md => deployments/Networking and Access/Sophos/Configure a Site-to-Site IPsec VPN.md (87%) rename reference/infrastructure/networking/Controllers/UniFi Controller.md => deployments/Networking and Access/UniFi/Deploy UniFi Controller with Docker.md (55%) rename reference/infrastructure/networking/Controllers/UniFi Network Server Controller.md => deployments/Networking and Access/UniFi/Deploy UniFi Network Server on Ubuntu.md (90%) create mode 100644 deployments/Networking and Access/index.md rename scripts/Bash/Install QEMU Guest Agent.md => deployments/Virtualization and Storage/Guests/Install the QEMU Guest Agent.md (65%) rename deployments/{platforms/virtualization/Hyper V => Virtualization and Storage/Hyper-V}/Failover Cluster/Deploy Failover Cluster Node.md (69%) rename deployments/{platforms/virtualization => Virtualization and Storage}/OpenStack/Ansible OpenStack.md (92%) rename deployments/{platforms/virtualization => Virtualization and Storage}/OpenStack/Canonical OpenStack.md (82%) rename deployments/{platforms/virtualization/proxmox => Virtualization and Storage/Proxmox}/Configuring ISCSI Based Cluster Storage.md (92%) rename deployments/{platforms/virtualization/proxmox/Cloud Init Templates/Ubuntu Server.md => Virtualization and Storage/Proxmox/Create an Ubuntu Cloud-Init Template.md} (86%) rename deployments/{platforms/virtualization/proxmox/ProxmoxVE.md => Virtualization and Storage/Proxmox/Deploy Proxmox VE.md} (80%) rename deployments/{platforms/virtualization/proxmox => Virtualization and Storage/Proxmox}/ZFS over ISCSI.md (89%) create mode 100644 deployments/Virtualization and Storage/Rancher Harvester/Harvester.md create mode 100644 deployments/Virtualization and Storage/index.md rename workflows/operations/Linux/Fedora Workstation/Full Setup.md => deployments/Windows and Linux/Fedora/Set Up the Fedora Workstation.md (90%) create mode 100644 deployments/Windows and Linux/index.md rename deployments/automation/{ansible/awx/Deployment => AWX}/AWX Operator.md (85%) rename deployments/automation/{ansible/awx/Deployment => AWX}/AWX in Minikube.md (66%) rename deployments/automation/{ansible/awx/awx.png => AWX/AWX.png} (100%) rename deployments/{services/documentation/docusaurus.md => automation/Documentation/Docusaurus.md} (57%) rename deployments/{services/documentation => automation/Documentation}/Material MkDocs.md (89%) rename deployments/{services/documentation/zensical.md => automation/Documentation/Zensical.md} (58%) rename deployments/{services/devops/gitea.md => automation/Gitea/Gitea.md} (75%) rename deployments/automation/{puppet/deployment => Puppet}/Puppet Bolt.md (88%) rename deployments/automation/{puppet/deployment/puppet.md => Puppet/Puppet.md} (92%) rename deployments/{services/rmm/tacticalrmm.md => automation/Remote Management/TacticalRMM.md} (74%) rename deployments/{services/Automation Tools => automation/Tools}/Activepieces.md (77%) create mode 100644 deployments/automation/Tools/Node Red.md rename deployments/{services/Automation Tools => automation/Tools}/Semaphore UI.md (82%) delete mode 100644 deployments/platforms/containerization/Docker/Custom Containers/Git Repo Updater.md delete mode 100644 deployments/platforms/index.md delete mode 100644 deployments/platforms/virtualization/Rancher Harvester/Harvester.md delete mode 100644 deployments/services/Automation Tools/Node Red.md delete mode 100644 deployments/services/DNS/AdGuard Home.md delete mode 100644 deployments/services/File Services/Windows Server/DFS Namespaces with Replication.md delete mode 100644 deployments/services/Security and Utility/Cyberchef.md delete mode 100644 deployments/services/Security and Utility/IT Tools.md delete mode 100644 deployments/services/authentication/keycloak/oauth2/Gitea OAuth2.md delete mode 100644 deployments/services/authentication/keycloak/oauth2/deployment.md delete mode 100644 deployments/services/cPanel/Creating Email Server.md delete mode 100644 deployments/services/email/Microsoft Exchange/Configuring ACME Letsencrypt Bot.md delete mode 100644 deployments/services/email/iRedMail/Quick Server Settings.md delete mode 100644 deployments/services/index.md delete mode 100644 deployments/services/productivity/OnlyOffice EE.md create mode 100644 reference/Applications/Email/iRedMail Connection Settings.md create mode 100644 reference/Applications/Email/index.md create mode 100644 reference/Applications/Files and Collaboration/Rclone Command Reference.md rename scripts/batch/robocopy.md => reference/Applications/Files and Collaboration/Robocopy Command Reference.md (88%) create mode 100644 reference/Applications/Files and Collaboration/index.md create mode 100644 reference/Applications/index.md create mode 100644 reference/Automation/AWX/Credential Configuration Examples.md rename workflows/operations/automation/ansible/credentials/Custom Credential Types/WinRM.md => reference/Automation/AWX/Custom Kerberos WinRM Credential.md (59%) rename workflows/operations/automation/ansible/inventories/overview.md => reference/Automation/AWX/Inventory Structure and Variables.md (74%) rename workflows/operations/automation/ansible/templates/overview.md => reference/Automation/AWX/Job Template Configuration.md (71%) rename workflows/operations/automation/ansible/playbooks/playbooks.md => reference/Automation/AWX/Playbook Catalog.md (83%) create mode 100644 reference/Automation/AWX/Projects and Source Control.md create mode 100644 reference/Automation/AWX/index.md create mode 100644 reference/Automation/Gitea Configuration Delivery.md create mode 100644 reference/Automation/index.md rename workflows/operations/Backups and DR/Veeam Backup Replication/Core Veeam Concepts.md => reference/Backup and Recovery/Veeam Concepts.md (95%) rename workflows/operations/Backups and DR/Veeam Backup Replication/Retention Best Practices.md => reference/Backup and Recovery/Veeam Retention Policy Example.md (74%) create mode 100644 reference/Backup and Recovery/index.md create mode 100644 reference/Containers/index.md rename deployments/services/authentication/Active Directory/LDAP Settings.md => reference/Identity and Certificates/Active Directory/LDAP Connection Settings.md (70%) create mode 100644 reference/Identity and Certificates/Keycloak Integrations.md create mode 100644 reference/Identity and Certificates/index.md rename reference/{infrastructure/hardware/Cluster Node 01/Disk Arrays.md => Lab Map/Hardware/Cluster Node 01 Disk Layout.md} (80%) rename reference/{infrastructure/hardware/Cluster Node 02/Disk Arrays.md => Lab Map/Hardware/Cluster Node 02 Disk Layout.md} (85%) rename reference/{infrastructure/hardware/Cluster Node 03/Disk Arrays.md => Lab Map/Hardware/Cluster Node 03 Disk Layout.md} (78%) rename reference/{infrastructure/hardware/Storage Node 01 Truenas Core/Disk Arrays.md => Lab Map/Hardware/Storage Node 01 TrueNAS Core Disk Layout.md} (88%) rename reference/{infrastructure/hardware/Storage Node 02 Truenas Scale/Disk Arrays.md => Lab Map/Hardware/Storage Node 02 TrueNAS Scale Disk Layout.md} (70%) rename workflows/operations/Hardware Management/ILO/Generic ILO Advanced License Keys.md => reference/Lab Map/Hardware/iLO License Reference.md (77%) rename reference/{infrastructure/networking/IP Tables => Lab Map}/Homelab Server Inventory.md (86%) rename reference/{infrastructure/networking/IP Tables => Lab Map/Network Address Plans}/172.16.16.0 Sophos Network.md (58%) rename reference/{infrastructure/networking/IP Tables => Lab Map/Network Address Plans}/192.168.5.0 Container Network.md (83%) rename workflows/operations/Power and UPS/APC Cell Wiring Diagram.md => reference/Lab Map/Power/APC Battery Cell Wiring.md (68%) rename workflows/operations/Power and UPS/Battery Backup Power Distribution.md => reference/Lab Map/Power/UPS Power Distribution.md (82%) create mode 100644 reference/Lab Map/index.md rename deployments/services/DNS/Windows Server/Best Practices.md => reference/Networking and Access/DNS/Windows DNS Configuration Notes.md (68%) create mode 100644 reference/Networking and Access/index.md create mode 100644 reference/Virtualization and Storage/Proxmox/index.md create mode 100644 reference/Virtualization and Storage/index.md rename scripts/Bash/Time Adjustment.md => reference/Windows and Linux/Linux Time Commands.md (57%) create mode 100644 reference/Windows and Linux/index.md create mode 100644 reference/foundations/Choose a Document Template.md create mode 100644 reference/foundations/Documentation Organization.md create mode 100644 reference/foundations/Documentation Path Changes.md delete mode 100644 reference/foundations/Templates/Document Template.md delete mode 100644 reference/infrastructure/hardware/index.md delete mode 100644 reference/infrastructure/networking/index.md rename scripts/{Powershell => Applications/Email}/Exchange Online/Connect to Exchange Online.md (60%) rename scripts/{Services => Applications}/Email/Microsoft Exchange/Set Mailbox Auto Reply.md (66%) create mode 100644 scripts/Applications/Email/Microsoft Exchange/Start Exchange Services.md create mode 100644 scripts/Applications/Files and Collaboration/DFS/Report DFS Namespaces and Replication.md create mode 100644 scripts/Applications/Files and Collaboration/DFS/Report DFS Replication Backlog.md rename scripts/{Powershell/General Purpose => Applications/Files and Collaboration}/Directory Walker.md (82%) rename scripts/{Powershell/General Purpose => Applications/Files and Collaboration}/File Finder.md (85%) rename scripts/{Powershell/SMB/Detailed Permission Report All Shares.md => Applications/Files and Collaboration/SMB/Report NTFS Permissions Across Shares.md} (77%) rename scripts/{Powershell/SMB/Top Level Permission Report All Shares.md => Applications/Files and Collaboration/SMB/Report SMB Share Permissions.md} (70%) rename scripts/{Powershell/Nextcloud/Upload Data to Nextcloud Share.md => Applications/Files and Collaboration/Upload Data to a Nextcloud Share.md} (92%) rename scripts/{Powershell/Minecraft Server/Update Script.md => Applications/Gaming and Media/Update the ATM10 Minecraft Server.md} (94%) rename scripts/{batch/Blue Iris/Server Watchdog.md => Applications/Home Automation/Blue Iris Server Watchdog.md} (61%) create mode 100644 scripts/Applications/index.md rename scripts/{Bash => Automation/Gitea}/Git Repo Updater.md (83%) create mode 100644 scripts/Automation/index.md delete mode 100644 scripts/Bash/Fix Displaylink Issues on Linux.md delete mode 100644 scripts/Bash/Mdadm Grow Array Size.md delete mode 100644 scripts/Bash/Open Port Checker.md delete mode 100644 scripts/Bash/Transfer Files with Netcat.md create mode 100644 scripts/Identity and Certificates/Active Directory/Force Group Policy Updates Across the Domain.md rename scripts/{Powershell/Reporting => Identity and Certificates/Active Directory/Reports}/Get Password Expiration.md (59%) create mode 100644 scripts/Identity and Certificates/Active Directory/Reports/Inactive Computers.md create mode 100644 scripts/Identity and Certificates/Active Directory/Reports/Inactive Users.md rename scripts/{Powershell/Azure => Identity and Certificates/Microsoft 365}/Check Email Aliases.md (50%) rename scripts/{Powershell/Azure => Identity and Certificates/Microsoft 365}/Connect to Azure AD.md (58%) create mode 100644 scripts/Identity and Certificates/index.md rename scripts/{Powershell/General Purpose/Remotely Change DNS Records.md => Networking and Access/Change DNS Client Server Settings Remotely.md} (62%) create mode 100644 scripts/Networking and Access/Check Open Ports.md rename scripts/{Powershell/General Purpose/DNS Hierarchy Correction.md => Networking and Access/Correct DNS Server Priority.md} (95%) create mode 100644 scripts/Networking and Access/index.md delete mode 100644 scripts/Powershell/General Purpose/Force GPUpdate Domain Wide.md delete mode 100644 scripts/Powershell/Reporting/Inactive Computers.md delete mode 100644 scripts/Powershell/Reporting/Inactive Users.md delete mode 100644 scripts/Services/Email/Microsoft Exchange/Restart Exchange Services.md rename scripts/{Powershell/Hyper V => Virtualization and Storage/Hyper-V}/Collapse Differencing Disk Chains.md (94%) rename scripts/{Powershell/Hyper V => Virtualization and Storage/Hyper-V}/Delete Locked VHDX File.md (67%) rename scripts/{Powershell/Hyper V => Virtualization and Storage/Hyper-V}/Failover Cluster/Force Reboot Remote Cluster Node.md (78%) rename scripts/{Powershell/Hyper V => Virtualization and Storage/Hyper-V}/Failover Cluster/Replication Bumper.md (88%) create mode 100644 scripts/Virtualization and Storage/Linux/Grow an mdadm Array.md rename scripts/{Bash/ProxmoxVE => Virtualization and Storage/Proxmox}/Deeplab Rollback Script.md (84%) create mode 100644 scripts/Virtualization and Storage/index.md rename scripts/{Powershell/General Purpose/Inactive User Profile Data Cleanup.md => Windows and Linux/Windows/Clean Up Inactive User Profiles.md} (98%) rename scripts/{Powershell/General Purpose/Fix Corrupted Windows Updates.md => Windows and Linux/Windows/Repair Windows Update Components.md} (79%) rename scripts/{Powershell/General Purpose/Restart Service Domain Wide.md => Windows and Linux/Windows/Start the RMM Agent Service Across the Domain.md} (77%) rename scripts/{Powershell/General Purpose/Windows 11 Upgrade via UNC Path.md => Windows and Linux/Windows/Upgrade Windows 11 from a UNC Path.md} (99%) create mode 100644 scripts/Windows and Linux/index.md rename deployments/services/communication/rocketchat/Autotask Regex Replacer.md => workflows/Applications/Communication/Rocketchat/Configure Autotask Link Replacement.md (75%) rename scripts/Services/Email/Microsoft Exchange/DAG/Database Management.md => workflows/Applications/Email/Microsoft Exchange/Manage DAG Database Copies.md (82%) rename {scripts/Services/Email/Microsoft Exchange/DAG => workflows/Applications/Email/Microsoft Exchange}/Perform Exchange SE DAG Rolling Updates.md (99%) rename deployments/services/email/Microsoft Exchange/Preparing for Cumulative Updates.md => workflows/Applications/Email/Microsoft Exchange/Prepare for Cumulative Updates.md (96%) create mode 100644 workflows/Applications/Email/Proxmox Mail Gateway/Repair Trusted Mail Delivery Between PMG and Mailcow.md rename deployments/services/email/iRedMail/Query SMTP Outgoing Queue.md => workflows/Applications/Email/iRedMail/Inspect the Outgoing SMTP Queue.md (84%) rename scripts/Powershell/General Purpose/Rclone.md => workflows/Applications/Files and Collaboration/Configure and Recover Rclone Bisync.md (83%) create mode 100644 workflows/Applications/Files and Collaboration/DFS/Repair an Inconsistent DFS Management Console.md rename deployments/services/Microsoft 365/Seize Control of Personal OneDrive Data of Another User.md => workflows/Applications/Files and Collaboration/Microsoft 365/Access Another User OneDrive Data.md (84%) create mode 100644 workflows/Applications/Files and Collaboration/Transfer Files with Netcat.md rename reference/infrastructure/networking/Misc/Tuya Smart Lights.md => workflows/Applications/Home Automation/Connect Tuya Smart Lights.md (61%) create mode 100644 workflows/Applications/index.md rename workflows/{operations/automation/ansible => Automation}/AWX/AWX Kerberos Implementation.md (92%) rename workflows/{operations/automation/ansible => Automation}/AWX/Connect AWX to Gitea.md (84%) rename deployments/automation/ansible/awx/Deployment/Upgrading Issues past 2 10 0.md => workflows/Automation/AWX/Repair Upgrades Beyond AWX Operator 2.10.0.md (87%) create mode 100644 workflows/Automation/Gitea/Deliver Configuration with a Docker Runner.md create mode 100644 workflows/Automation/Gitea/Publish Zensical Documentation with a Host Runner.md create mode 100644 workflows/Automation/index.md rename workflows/{operations/Backups and DR/Veeam Backup Replication => Backup and Recovery/Veeam}/Backup Agent Takeover.md (52%) rename workflows/{operations/Backups and DR/Veeam Backup Replication/Troubleshooting => Backup and Recovery/Veeam}/Failed to Validate Certificates of Some Gateways.md (85%) rename workflows/{operations/Backups and DR/Veeam Backup Replication => Backup and Recovery/Veeam}/Manually Pruning Backups.md (89%) rename workflows/{operations/Backups and DR/Veeam Backup Replication => Backup and Recovery/Veeam}/Migrating VSPC Backup Repositories.md (80%) create mode 100644 workflows/Backup and Recovery/index.md rename deployments/platforms/containerization/Docker/Custom Containers/Container Development.md => workflows/Containers/Docker/Build and Publish a Container Image.md (96%) rename reference/infrastructure/networking/Docker Networking/Creating a Macvlan Sub Interface for Docker.md => workflows/Containers/Docker/Create a Macvlan Subinterface.md (84%) rename scripts/Bash/Transfer Docker Containers.md => workflows/Containers/Docker/Transfer Docker Containers Between Hosts.md (88%) rename workflows/{Platforms/Containerization => Containers}/Kubernetes/Migrating Docker Compose YML to K8s.md (93%) create mode 100644 workflows/Containers/index.md rename deployments/services/authentication/Active Directory/Group Policy/Desktop Shortcut to UNC Path.md => workflows/Identity and Certificates/Active Directory/Create a Desktop Shortcut to a UNC Path.md (79%) rename {deployments/services/authentication => workflows/Identity and Certificates}/Active Directory/Restore Domain Trust.md (62%) rename workflows/{operations/Windows/Windows Server/SSL Certificates/Convert SSL Certificates into PFX Files.md => Identity and Certificates/Certificates/Convert Certificates to PFX.md} (69%) create mode 100644 workflows/Identity and Certificates/Certificates/Export Certificates for LDAPS Clients.md create mode 100644 workflows/Identity and Certificates/Certificates/Publish and Maintain Certificate Revocation Lists.md create mode 100644 workflows/Identity and Certificates/Keycloak/Connect Gitea to Keycloak.md rename deployments/services/authentication/keycloak/oauth2/Portainer OAuth2.md => workflows/Identity and Certificates/Keycloak/Connect Portainer to Keycloak.md (52%) rename {deployments/services => workflows/Identity and Certificates}/Microsoft 365/Change MFA Settings.md (78%) rename workflows/{operations/automation/ansible/Enable WinRM on Windows Devices.md => Identity and Certificates/Windows/Enable WinRM over HTTPS.md} (93%) create mode 100644 workflows/Identity and Certificates/index.md rename reference/infrastructure/networking/Linux Networking/Modifying IP Address of Server.md => workflows/Networking and Access/Linux/Change a Server IP Address.md (62%) rename {scripts/Bash => workflows/Networking and Access/Linux}/Configure SSH Key Authentication.md (85%) rename reference/infrastructure/networking/Firewall and Routing/Sophos/VPN/Site to Site VPNs/IPSEC/Automatic Tunnel Resetting.md => workflows/Networking and Access/Sophos/Automatically Reset an IPsec Tunnel.md (91%) rename {reference/infrastructure/networking/Firewall and Routing => workflows/Networking and Access}/Sophos/Configure LAN Bridging.md (78%) rename reference/infrastructure/networking/Firewall and Routing/Sophos/VPN/SSL VPN/Configuring Remote VPN RDP Access.md => workflows/Networking and Access/Sophos/Configure RDP Access over SSL VPN.md (89%) create mode 100644 workflows/Networking and Access/index.md delete mode 100644 workflows/Platforms/Virtualization/Proxmox/Common Tasks.md rename workflows/{Platforms/Virtualization/Hyper V => Virtualization and Storage/Hyper-V}/Failover Cluster/Rebuild Failover Cluster Replication.md (83%) rename workflows/{Platforms/Virtualization/Hyper V => Virtualization and Storage/Hyper-V}/Forcefully Stop GuestVM.md (80%) rename workflows/{Platforms/Virtualization/Hyper V => Virtualization and Storage/Hyper-V}/Kerberos Enabled VM Migration.md (89%) rename workflows/{operations/Linux/Expanding Linux Filesystems.md => Virtualization and Storage/Linux/Expand a Linux Guest Filesystem.md} (95%) rename workflows/{operations/Linux/Expand ISCSI Based ZFS Filesystem.md => Virtualization and Storage/Linux/Expand an iSCSI-Backed ZFS Filesystem.md} (90%) rename deployments/platforms/virtualization/proxmox/Detecting and Removing Orphaned VM Disks.md => workflows/Virtualization and Storage/Proxmox/Detect and Remove Orphaned VM Disks.md (87%) rename workflows/{Platforms/Virtualization/Proxmox/Operations/Manually Activate Volume Group.md => Virtualization and Storage/Proxmox/Manually Activate a Volume Group.md} (72%) create mode 100644 workflows/Virtualization and Storage/Proxmox/Remove a Node from a Proxmox Cluster.md rename workflows/{operations/Backups and DR/Veeam Backup Replication/Migrating VMs to ProxmoxVE.md => Virtualization and Storage/Proxmox/Repair Rocky Linux After a Veeam Migration.md} (93%) rename deployments/platforms/virtualization/proxmox/Fixing iSCSI Connections that Drop at Reboot.md => workflows/Virtualization and Storage/Proxmox/Repair iSCSI Connections After Reboot.md (84%) rename workflows/{Platforms/Virtualization/Proxmox/Operations/Upgrade PVE from 8 to 9.md => Virtualization and Storage/Proxmox/Upgrade Proxmox VE from 8 to 9.md} (86%) rename reference/infrastructure/hardware/Storage Node 01 Truenas Core/Replacing a Drive.md => workflows/Virtualization and Storage/TrueNAS/Replace a Drive in Storage Node 01.md (76%) create mode 100644 workflows/Virtualization and Storage/index.md rename workflows/{operations => Windows and Linux}/Linux/CachyOS/Restrict Monitors on Plasma Login Screen.md (78%) rename workflows/{operations => Windows and Linux}/Linux/Fedora Workstation/Install Flatpak Apps.md (60%) rename workflows/{operations => Windows and Linux}/Linux/Fedora Workstation/Upgrading Versions.md (66%) create mode 100644 workflows/Windows and Linux/Linux/Repair DisplayLink USB Authorization.md rename workflows/{operations => Windows and Linux}/Windows/Change Windows Edition.md (74%) rename workflows/{operations => Windows and Linux}/Windows/Delete Windows Recovery Partition.md (91%) rename workflows/{operations => Windows and Linux}/Windows/Uninstall Updates via DISM.md (89%) create mode 100644 workflows/Windows and Linux/Windows/VSS/Delete Shadow Copies.md rename workflows/{operations => Windows and Linux}/Windows/Windows 11/Creating a Local Account on Win11.md (65%) create mode 100644 workflows/Windows and Linux/index.md delete mode 100644 workflows/operations/Windows/VSS/Delete Shadow Copies.md delete mode 100644 workflows/operations/automation/ansible/credentials/overview.md delete mode 100644 workflows/operations/automation/ansible/projects/overview.md diff --git a/blog/index.md b/blog/index.md index 2f78c81..7002411 100644 --- a/blog/index.md +++ b/blog/index.md @@ -1,26 +1,16 @@ --- tags: - Blog - - Index - Documentation --- # Blog ## Purpose -Narrative posts for lessons learned, experiments, and updates. +Read the dated experiments and lessons behind changes in the lab. Follow the linked operational pages when you need the reusable configuration or procedure. -## New Post Template -````markdown -# -## Context - +## Includes +- Posts -## What Changed -- - -## Results -- - -## Lessons Learned -- -```` +## Follow the Experiments +- [Gitea Runners]() — Read the configuration-delivery experiment and continue to the extracted runner procedures. +- [OpenStack]() — Read the context behind the separate OpenStack deployment attempts. diff --git a/blog/posts/01-22-2025 Windows Power Profiles Causing Notable CPU Performance Loss.md b/blog/posts/01-22-2025 Windows Power Profiles Causing Notable CPU Performance Loss.md index 253872a..c4a0835 100644 --- a/blog/posts/01-22-2025 Windows Power Profiles Causing Notable CPU Performance Loss.md +++ b/blog/posts/01-22-2025 Windows Power Profiles Causing Notable CPU Performance Loss.md @@ -1,5 +1,5 @@ --- -draft: false +draft: false date: 2025-01-22 updated: 2025-01-22 authors: @@ -20,12 +20,15 @@ So I've been noticing a trend recently regarding something I never really took m ## Overview of the Problem The general idea is that Windows devices (Workstations & Servers) have what are called power "**profiles**". These profiles, by default, are set to "**Balanced**". Which in basic terms means that the operating system will artificially limit the CPU speed to below 2.0GHz at all times. This means if the CPU is capable of 4GHz, it will be limited to 2GHz no-matter-what. This is a huge problem since it leaves performance just sitting on the table. -## Observations & Actions Taken -When I learned of the above, I began to audit every Windows-based server and workstation (Physical and Virtual) in my homelab. The virtual machines seemed unaffected by this issue, but I still configured them to "**High Performance**" power profiles regardless. However, every single physical host (`VIRT-NODE-01`, `VIRT-NODE-02`, and `LAB-DRAAS-01`), all saw notable performance improvements ranging from 32% to 41%, on average going from 1.75GHz to 2.6GHz on the virtualization hosts, and 1.9GHz to 3.2GHz on the backup server. +## Observations and Actions Taken +When I learned of the above, I began to audit every Windows-based server and workstation (Physical and Virtual) in my homelab. The virtual machines seemed unaffected by this issue, but I still configured them to "**High Performance**" power profiles regardless. However, every single physical host (`VIRT-NODE-01`, `VIRT-NODE-02`, and `LAB-DRAAS-01`), all saw notable performance improvements ranging from 32% to 41%, on average going from 1.75GHz to 2.6GHz on the virtualization hosts, and 1.9GHz to 3.2GHz on the backup server. ## Final Thoughts -I am so upset that for so many years, it never occured to me that the power profiles applied to server operating systems. I always just *assumed* they ran in "**High Performance**" power profiles all the time. I discovered I had non-trivial amounts of performance loss because of this simple checkbox setting in the OS. - +I am so upset that for so many years, it never occured to me that the power profiles applied to server operating systems. I always just *assumed* they ran in "**High Performance**" power profiles all the time. I discovered I had non-trivial amounts of performance loss because of this simple checkbox setting in the OS. !!! success "Performance Improvements" - The two Hyper-V Failover Cluster hosts saw a **32%** performance improvement (**1.75GHz to 2.6GHz**), while the Veeam Backup & Replication Server host observed a whopping **42%** performance improvement (**1.9GHz to 3.2GHz**). \ No newline at end of file + The two Hyper-V Failover Cluster hosts saw a **32%** performance improvement (**1.75GHz to 2.6GHz**), while the Veeam Backup & Replication Server host observed a whopping **42%** performance improvement (**1.9GHz to 3.2GHz**). + +## Related Documentation +- [Windows and Linux Administration](<../../reference/Windows and Linux/index.md>) — Find the operating-system procedures related to this performance observation. +- [Virtualization and Storage](<../../reference/Virtualization and Storage/index.md>) — Find the host and guest documentation for the environment being tuned. diff --git a/blog/posts/05-16-2025 Learning to Leverage Gitea Runners.md b/blog/posts/05-16-2025 Learning to Leverage Gitea Runners.md index 2afef51..4dc8878 100644 --- a/blog/posts/05-16-2025 Learning to Leverage Gitea Runners.md +++ b/blog/posts/05-16-2025 Learning to Leverage Gitea Runners.md @@ -1,7 +1,7 @@ --- -draft: false +draft: false date: 2025-05-16 -updated: 2025-05-16 +updated: 2026-09-05 authors: - nicole categories: @@ -15,7 +15,7 @@ tags: --- # Learning to Leverage Gitea Runners -When I first started my journey with a GitOps mentality to transition a portion of my homelab's infrastructure to an "**Intrastructure-as-Code**" structure, I had made my own self-made Docker container that I called the [Git-Repo-Updater](../../deployments/platforms/containerization/docker/custom-containers/git-repo-updater.md). This self-made tool was useful to me because it copied the contents of Gitea repositories into bind-mounted container folders on my Portainer servers. This allowed me to set up configurations for Homepage-Docker, Material MkDocs, Traefik Reverse Proxy, and others to pull configuration changes from Gitea directly into the production servers, causing them to hot-load the changes instantly. (within 10 seconds, give or take). +When I first started my journey with a GitOps mentality to transition a portion of my homelab's infrastructure to an "**Intrastructure-as-Code**" structure, I had made my own self-made Docker container that I called the [Git-Repo-Updater](<../../deployments/Containers/Docker/Git Repo Updater.md>). This self-made tool was useful to me because it copied the contents of Gitea repositories into bind-mounted container folders on my Portainer servers. This allowed me to set up configurations for Homepage-Docker, Material MkDocs, Traefik Reverse Proxy, and others to pull configuration changes from Gitea directly into the production servers, causing them to hot-load the changes instantly. (within 10 seconds, give or take). ## Criticisms of Git-Repo-Updater When I made the [Git-Repo-Updater docker container stack](https://git.bunny-lab.io/container-registry/git-repo-updater), I ran into the issue of having made something I knew existing solutions existed for but simply did not understand well-enough to use yet. This caused me to basically delegate the GitOps workflow to a bash script with a few environment variables, running inside of an Alpine Linux container. While the container did it's job, it would occassionally have hiccups, caching issues, or repository branch errors that made no sense. This lack of transparency and the need to build an entire VSCode development environment to push new docker package updates to Gitea's [package repository for Git-Repo-Updater](https://git.bunny-lab.io/container-registry/-/packages/container/git-repo-updater/latest) caused a lot of development headaches. @@ -31,80 +31,10 @@ When I finally got around to figuring out the general architecture of how [Gitea - The Gitea repository has a Gitea-specific workflows folder that holds `.yaml` files that the runner uses to define tasks that occur when the repository has changes made to it / commits pushed to it. - The runner checks out the repository (*clones it to the runner environment*), and leverages rsync to copy the data into the production server's configuration folder(s) based on the unique needs of the task. - The runner cleans up after itself and returns back to an "Idle" state. -- The production server hot-loads the changed configuration files (e.g. Material MkDocs, Traefik, Nginx, etc) and the changes go to into effect immediately +- The production server hot-loads the changed configuration files (e.g. Material MkDocs, Traefik, Nginx, etc) and the changes go to into effect immediately -### Docker-Compose Runner Deployment -When it comes to deploying a runner, (*assuming you want to use a docker-based runner*) it has a few simple things that need to be configured, the `docker-compose.yml` and the `.env` files. These tell the runner to reach out to Gitea server to register the runner with the given repository that you generated a registration token on. - -```yaml title="docker-compose.yml" -version: "3.8" -services: - app: - image: docker.io/gitea/act_runner:latest - environment: - CONFIG_FILE: /config.yaml - GITEA_INSTANCE_URL: "${INSTANCE_URL}" - GITEA_RUNNER_REGISTRATION_TOKEN: "${REGISTRATION_TOKEN}" - GITEA_RUNNER_NAME: "${RUNNER_NAME}" - GITEA_RUNNER_LABELS: "${RUNNER_NAME}" # This can be anything, and is referenced by the workflow task(s) later. - volumes: - - /srv/containers/gitea-runner-mkdocs/config.yaml:/config.yaml # You have to manually make this file before you start the container - - /srv/containers/material-mkdocs/docs/docs:/Gitops_Destination # This is where the repository data will be copied to -``` - -```sh title=".env" -INSTANCE_URL=https://git.bunny-lab.io -RUNNER_NAME=gitea-runner-mkdocs -REGISTRATION_TOKEN= -``` - -### Creating the `config.yaml` -The oddball thing about the way that I configured the Gitea Act Runner was telling it to run the container in "host mode" which tells it to run the tasks / workflows directly on the container itself instead of spinning up an instanced container (referred to as "*Docker-in-Docker*"). This keeps things simpler, but requires us to add a line to the `config.yaml` located at `/srv/containers/gitea-runner-mkdocs/config.yaml`. You can use your preferred text editor to add the following to the file's contents. This tells the runner to use itself for the tasks instead of an instanced docker container. -```yaml title="/srv/containers/gitea-runner-mkdocs/config.yaml" -container_engine: "" -``` - -!!! info "Quick Config Command" - ```sh - mkdir -p /srv/containers/gitea-runner-mkdocs - echo 'container_engine: ""' > "/srv/containers/gitea-runner-mkdocs/config.yaml" - ``` - -### Runner Workflow Task Files -When it comes to telling the runner what to do and how to do it, you create what are called runner "**Workflows**". These files reside within `/.gitea/workflows` and are `.yaml` format. If you have any familiarity with Ansible, the similarities are staggaring. You can have multiple workflows for one repository, with different flows that fire-off on different runners. An example of the flow used to replace Git-Repo-Updater's functionality can be seen below. - - -In the workflow below, it spins up a runner within the Alpine Linux environment that the `docker.io/gitea/act_runner:latest` uses, then installs NodeJS, Git, and Rsync for the core functionality that mirrors Git-Repo-Updater: -```yaml title=".gitea/workflows/gitops-automatic-deployment.yml" -name: GitOps Automatic Deployment - -on: - push: - branches: [ main ] - -jobs: - GitOps Automatic Deployment: - runs-on: gitea-runner-mkdocs - - steps: - - name: Install Node.js, git, rsync, and curl - run: | - apk add --no-cache nodejs npm git rsync curl - - - name: Checkout Repository - uses: actions/checkout@v3 - - - name: Copy Repository Data to Production Server - run: | - rsync -a --delete --exclude='.git/' --exclude='.gitea/' . /Gitops_Destination/ - - - name: Notify via NTFY - run: | - curl -d "https://docs.bunny-lab.io - Workflow Completed" https://ntfy.bunny-lab.io/gitea-runners -``` - -!!! note "`runs-on` Variable" - In this example workflow file, we are targeting the previously-mentioned `gitea-runner-mkdocs` runner, which we gave that "label" in the docker-compose.yaml file's `GITEA_RUNNER_LABELS` variable. You can name these labels whatever you want, as a way of organizing which runners run which workflows associated with a repository when changes are made to the repository. +### Runner Configuration +I have separated the Docker runner configuration and repository workflow into [Deliver Configuration with a Docker Runner](<../../workflows/Automation/Gitea/Deliver Configuration with a Docker Runner.md>). This preserves the operational example alongside the reasons I chose it. The [Zensical host runner workflow](<../../workflows/Automation/Gitea/Publish Zensical Documentation with a Host Runner.md>) documents the separate host-based implementation. ### It All Comes Together When all of this is set up, it works exactly like Git-Repo-Updater did, but more securely, faster, and more robustly, as the tasks can be changed from the repository-level instead of having to make changes inside of a `Dockerfile` or having to learn how to publish your own Docker containers to a registry. When you learn how to do it, it becomes faster and easier to set up than Git-Repo-Updater as well. @@ -113,6 +43,4 @@ Then, when you push changes to a repository, the workflow's task triggers automa Gitea Act Runners are a beautiful thing, and it's a damn shame it took me this long to get around to learning how they work and using them. -![Gitea_Runner_Screenshot](./Images/Gitea_Runner_Screenshot.png) - - +![Gitea_Runner_Screenshot]() diff --git a/blog/posts/12-15-2024 Implementing the Blog Plugin.md b/blog/posts/12-15-2024 Implementing the Blog Plugin.md index 6b3a1f5..151c1ae 100644 --- a/blog/posts/12-15-2024 Implementing the Blog Plugin.md +++ b/blog/posts/12-15-2024 Implementing the Blog Plugin.md @@ -1,5 +1,5 @@ --- -draft: false +draft: false date: 2024-12-15 updated: 2024-12-15 authors: @@ -39,4 +39,8 @@ tags: # Example Post Title Placeholder Text / Body of the Blog Post. -``` \ No newline at end of file +``` + +## Related Documentation +- [Zensical Deployment](<../../deployments/automation/Documentation/Zensical.md>) — Find the separately documented Zensical service and its content-delivery workflow. +- [Blog Styling Contract](<../../reference/foundations/Documentation Styling.md>) — Use the canonical blog front matter and narrative structure. diff --git a/blog/posts/12-15-2024 OpenStack Frustrations.md b/blog/posts/12-15-2024 OpenStack Frustrations.md index ef10c0e..ddafc2a 100644 --- a/blog/posts/12-15-2024 OpenStack Frustrations.md +++ b/blog/posts/12-15-2024 OpenStack Frustrations.md @@ -1,12 +1,12 @@ --- -draft: false +draft: false date: 2024-12-15 updated: 2024-12-15 authors: - nicole links: - - platforms/virtualization/openstack/ansible-openstack.md - - platforms/virtualization/openstack/canonical-openstack.md + - ../../deployments/Virtualization and Storage/OpenStack/Ansible OpenStack.md + - ../../deployments/Virtualization and Storage/OpenStack/Canonical OpenStack.md categories: - Virtualization - Containers @@ -22,12 +22,10 @@ tags: So, I want to start with a little context. As part of a long-standing project I have been working on, I have tried to deploy OpenStack. OpenStack is sort of envisioned as "Infrastructure as a Service (IAAS)". Basically you deploy an OpenStack cluster, which can run its own KVM for virtual machine and containers, or it can interface with an existing Hypervisor infrastructure, such as Hyper-V. In most cases, people branch out the "Control", "Compute", and "Storage" roles into different physical servers, but in my homelab, I have been attempting to deploy it via a "Converged" model, of having Control, Compute, and Storage on each node, spanning a high-availability cluster of 3 nodes. ## The Problem -The problems come into the overall documentation provided for deploying either [Canonical Openstack](https://ubuntu.com/openstack/install) which I have detailed my frustrations of the system in my own attempted re-write of the documentation [here](../../deployments/platforms/virtualization/openstack/canonical-openstack.md). I have also attempted to deploy it via [Ansible OpenStack](https://docs.openstack.org/project-deploy-guide/openstack-ansible/2024.1/), whereas my documentation thus far in my homelab is visible [here](../../deployments/platforms/virtualization/openstack/ansible-openstack.md). +The problems come into the overall documentation provided for deploying either [Canonical Openstack](https://ubuntu.com/openstack/install) which I have detailed my frustrations of the system in my own attempted re-write of the documentation [here](<../../deployments/Virtualization and Storage/OpenStack/Canonical OpenStack.md>). I have also attempted to deploy it via [Ansible OpenStack](https://docs.openstack.org/project-deploy-guide/openstack-ansible/2024.1/), whereas my documentation thus far in my homelab is visible [here](<../../deployments/Virtualization and Storage/OpenStack/Ansible OpenStack.md>). You see, OpenStack is like icecream, it has many different ways to deploy it, and it can be as simple, or as overtly-complex as you need it to be, and it scales *really well* across a fleet of servers in a datacenter. My problems come in where the Canonical deployment has never worked fully / properly, and their own development team is hesitant to recommend the current documentation, and the Ansible OpenStack deployment process, while relatively simple, requires a base of existing knowledge that makes translating the instructions into more user-friendly instructions in my homelab documentation a difficult task. Eventually I want to automate much of the process as much as I can, but that will take time. The common issue I've seen while trying to deploy OpenStack is understanding the networking, how networking is configured, network bridges, etc. The process is different based on the deployment method (Currently trying to deploy it via OpenStack Ansible). Hopefully in the near future I will make some kind of breakthrough in the deployment process and get everything working. I will post an update later if I figure things out! - - diff --git a/deployments/services/Asset Management/Homebox.md b/deployments/Applications/Asset Management/Homebox.md similarity index 71% rename from deployments/services/Asset Management/Homebox.md rename to deployments/Applications/Asset Management/Homebox.md index 1590407..45ddf62 100644 --- a/deployments/services/Asset Management/Homebox.md +++ b/deployments/Applications/Asset Management/Homebox.md @@ -5,14 +5,15 @@ tags: - Docker --- -**Purpose**: Homebox is the inventory and organization system built for the Home User! With a focus on simplicity and ease of use, Homebox is the perfect solution for your home inventory, organization, and management needs. +## Purpose +Homebox is the inventory and organization system built for the Home User! With a focus on simplicity and ease of use, Homebox is the perfect solution for your home inventory, organization, and management needs. [Reference Documentation](https://hay-kot.github.io/homebox/quick-start/) !!! warning "Protect with Keycloak" - The GitHub project for this software appears to have been archived in a read-only state in June 2024. There is no default admin credential, so setting the environment variable `HBOX_OPTIONS_ALLOW_REGISTRATION` to `false` will literally make you unable to log into the system. You also cannot change it after-the-fact, so you cannot just register an account then disable it and restart the container, it doesn't work that way. + The GitHub project for this software appears to have been archived in a read-only state in June 2024. There is no default admin credential, so setting the environment variable `HBOX_OPTIONS_ALLOW_REGISTRATION` to `false` will literally make you unable to log into the system. You also cannot change it after-the-fact, so you cannot just register an account then disable it and restart the container, it doesn't work that way. - Due to this behavior, it is imperative that you deploy this either only internally, or if its external, put it behind something like [Authentik](../authentication/authentik.md) or [Keycloak](../authentication/keycloak/deployment.md). + Due to this behavior, it is imperative that you deploy this either only internally, or if its external, put it behind something like [Authentik](<../../Identity and Certificates/Authentik.md>) or [Keycloak](<../../Identity and Certificates/Keycloak/Deploy Keycloak.md>). ## Docker Configuration ```yaml title="docker-compose.yml" @@ -60,6 +61,7 @@ HBOX_MAILER_FROM=noreply@bunny-lab.io ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -81,3 +83,7 @@ http: passHostHeader: true ``` +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/Asset Management/Snipe IT.md b/deployments/Applications/Asset Management/Snipe IT.md similarity index 84% rename from deployments/services/Asset Management/Snipe IT.md rename to deployments/Applications/Asset Management/Snipe IT.md index 757b865..548dc49 100644 --- a/deployments/services/Asset Management/Snipe IT.md +++ b/deployments/Applications/Asset Management/Snipe IT.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: A free open source IT asset/license management system. +## Purpose +A free open source IT asset/license management system. !!! warning The Snipe-IT container will attempt to launch after the MariaDB container starts, but MariaDB takes a while set itself up before it can accept connections; as a result, Snipe-IT will fail to initialize the database. Just wait about 30 seconds after deploying the stack, then restart the Snipe-IT container to initialize the database. You will know it worked if you see notes about data being `Migrated`. @@ -111,6 +112,7 @@ APP_TRUSTED_PROXIES=192.168.5.29 ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -124,7 +126,7 @@ http: middlewares: - "assets-bunny-lab-io" - "auth-bunny-lab-io" # Referencing the Keycloak Server - + middlewares: assets-bunny-lab-io: headers: @@ -140,4 +142,9 @@ http: servers: - url: "http://192.168.5.50:8080" passHostHeader: true -``` \ No newline at end of file +``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/communication/niltalk.md b/deployments/Applications/Communication/Niltalk.md similarity index 64% rename from deployments/services/communication/niltalk.md rename to deployments/Applications/Communication/Niltalk.md index a086537..0f88ddc 100644 --- a/deployments/services/communication/niltalk.md +++ b/deployments/Applications/Communication/Niltalk.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Niltalk is a web based disposable chat server. It allows users to create password protected disposable, ephemeral chatrooms and invite peers to chat rooms. +## Purpose +Niltalk is a web based disposable chat server. It allows users to create password protected disposable, ephemeral chatrooms and invite peers to chat rooms. ```yaml title="docker-compose.yml" version: "3.7" @@ -50,3 +51,7 @@ volumes: ```yaml title=".env" Not Applicable ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/communication/rocketchat/deployment.md b/deployments/Applications/Communication/Rocketchat/Deploy Rocket.Chat.md similarity index 81% rename from deployments/services/communication/rocketchat/deployment.md rename to deployments/Applications/Communication/Rocketchat/Deploy Rocket.Chat.md index b8c1c5f..e1bec70 100644 --- a/deployments/services/communication/rocketchat/deployment.md +++ b/deployments/Applications/Communication/Rocketchat/Deploy Rocket.Chat.md @@ -5,11 +5,13 @@ tags: - Docker --- -**Purpose**: Deploy a RocketChat and MongoDB database together. +## Purpose +Deploy a RocketChat and MongoDB database together. -!!! caution Folder Pre-Creation - You need to make the folders for the Mongo database before launching the container stack for the first time. If you do not make this folder ahead of time, Mongo will give Permission Denied errors to the data directorry. You can create the folder as well as adjust permissions with the following commands: - ``` sh +!!! caution "Folder Pre-Creation" + You need to make the folders for the Mongo database before launching the container stack for the first time. If you do not make this folder ahead of time, Mongo will give Permission Denied errors to the data directorry. You can create the folder as well as adjust permissions with the following commands: + + ```sh mkdir -p /srv/containers/rocketchat/mongodb/data chmod -R 777 /srv/containers/rocketchat ``` @@ -51,7 +53,7 @@ services: networks: docker_network: ipv4_address: 192.168.5.2 - + rc_mongodb: image: docker.io/bitnami/mongodb:${MONGODB_VERSION:-5.0} restart: always @@ -69,7 +71,7 @@ services: networks: docker_network: ipv4_address: 192.168.5.3 - + networks: docker__network: external: true @@ -83,6 +85,7 @@ MONGODB_VERSION=6.0 MONGODB_INITIAL_PRIMARY_HOST=rc_mongodb #Redundant - Can be Removed MONGODB_ADVERTISED_HOSTNAME=rc_mongodb #Redundant - Can be Removed ``` + ## Reverse Proxy Configuration ```yaml title="nginx.conf" # Rocket.Chat Server @@ -105,3 +108,8 @@ server { } } ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/dashboards/dashy.md b/deployments/Applications/Dashboards/Dashy.md similarity index 70% rename from deployments/services/dashboards/dashy.md rename to deployments/Applications/Dashboards/Dashy.md index 462efa6..8028ab5 100644 --- a/deployments/services/dashboards/dashy.md +++ b/deployments/Applications/Dashboards/Dashy.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: A self-hostable personal dashboard built for you. Includes status-checking, widgets, themes, icon packs, a UI editor and tons more! +## Purpose +A self-hostable personal dashboard built for you. Includes status-checking, widgets, themes, icon packs, a UI editor and tons more! ```yaml title="docker-compose.yml" version: "3.8" @@ -32,7 +33,7 @@ services: - NODE_ENV=production - UID=1000 - GID=1000 - + # Pass in your config file below, by specifying the path on your host machine volumes: - /srv/Containers/Dashy/conf.yml:/app/public/conf.yml @@ -48,8 +49,8 @@ services: timeout: 10s retries: 3 start_period: 40s - - # Connect container to Docker_Network + + # Connect container to Docker_Network networks: docker_network: ipv4_address: 192.168.5.57 @@ -61,6 +62,6 @@ networks: external: true ``` -```jsx title=".env" -Not Applicable -``` +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/dashboards/Homepage Docker.md b/deployments/Applications/Dashboards/Homepage Docker.md similarity index 71% rename from deployments/services/dashboards/Homepage Docker.md rename to deployments/Applications/Dashboards/Homepage Docker.md index dd5f8b3..d4c0cf2 100644 --- a/deployments/services/dashboards/Homepage Docker.md +++ b/deployments/Applications/Dashboards/Homepage Docker.md @@ -5,7 +5,8 @@ tags: - Dashboards --- -**Purpose**: A highly customizable homepage (or startpage / application dashboard) with Docker and service API integrations. +## Purpose +A highly customizable homepage (or startpage / application dashboard) with Docker and service API integrations. ```yaml title="docker-compose.yml" version: '3.8' @@ -33,7 +34,7 @@ services: - "rancher.bunny-lab.io:192.168.3.21" networks: docker_network: - ipv4_address: 192.168.5.44 + ipv4_address: 192.168.5.44 dockerproxy: image: ghcr.io/tecnativa/docker-socket-proxy:latest @@ -50,8 +51,8 @@ services: restart: unless-stopped networks: docker_network: - ipv4_address: 192.168.5.46 - + ipv4_address: 192.168.5.46 + networks: default: external: @@ -63,3 +64,7 @@ networks: ```yaml title=".env" Not Applicable ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/Applications/Email/Microsoft Exchange/Configuring ACME Letsencrypt Bot.md b/deployments/Applications/Email/Microsoft Exchange/Configuring ACME Letsencrypt Bot.md new file mode 100644 index 0000000..e91aa7c --- /dev/null +++ b/deployments/Applications/Email/Microsoft Exchange/Configuring ACME Letsencrypt Bot.md @@ -0,0 +1,79 @@ +--- +tags: + - Microsoft Exchange + - Lets Encrypt + - Email +--- + +## Purpose +If you want to set up automatic Let's Encrypt SSL certificates on a Microsoft Exchange server, you have to go through a few steps to install the WinACME bot, and configure it to automatically renew certificates. + +!!! note "ACME Bot Provisioning Considerations" + This document assumes you want a fully-automated one-liner command for configuring the ACME Bot, it is also completely valid to go step-by-step through the bot to configure the SSL certificate, the IIS server, etc, and it will automatically create a Scheduled Task to renew on its own. The whole process is very straight-forward with most answers being the default option. + +### Download the Win-ACME Bot +- Log into the on-premise Exchange Server via Datto RMM +- Navigate to: [https://www.win-acme.com/](https://www.win-acme.com/) + - On the top-right of the website, you will see a "**Download**" button with the most recent version of the Win-ACME bot +- Extract the contents of the ZIP file to "**C:\\Program Files (x86)\\Lets Encrypt**" + - Make the "**Lets Encrypt**" folder if it does not already exist + +### Configure `settings_default.json` +- The next step involves us making a modification to the configuration of the Win-ACME bot that allows us to export the necessary private key data for Exchange +- Using a text editor, open the "**settings\_default.json**" file + - Look for the setting called "**PrivateKeyExportable**" and change the value from "**false**" to "**true**" + - Save and close the file + +### Download and Install the SSL Certificate +- Open an administrative Command Line (DO NOT USE POWERSHELL) +- Navigate to the Let's Encrypt bot directory: `CD "C:\Program Files (x86)\Lets Encrypt"` +- Invoke the bot to automatically download and install the certificate into the IIS Server that Exchange uses to host the Exchange Server + - Be sure to change the placeholder subdomains to match the domain of the actual Exchange Server + - (e.g. "**mail.example.org**" | "**autodiscover.example.org**") + + ```text + wacs.exe --target manual --host mail.example.org,autodiscover.example.org --certificatestore My --acl-fullcontrol "network service,administrators" --installation iis,script --installationsiteid 1 --script "./Scripts/ImportExchange.ps1" --scriptparameters "'{CertThumbprint}' 'IIS,SMTP,IMAP' 1 '{CacheFile}' '{CachePassword}' '{CertFriendlyName}'" --verbose + ``` + +- When the command is running, it will ask for an email address for alerts and abuse notifications, just put "**infrastructure@bunny-lab.io**" +- If you run into any unexpected errors that result in anything other than exiting with a status "0", consult with Nicole Rappe to proceed + - Check that the domain of the Exchange Server is reachable on port 80 as Let's Encrypt uses this to build the cert. + - Searching the external IP of the server on [Shodan](https://www.shodan.io/) will reveal all open ports. + +### Troubleshooting +If you find that any of the services such as [https://mail.example.org/ecp](https://mail.example.org/ecp), [https://autodiscover.example.org](https://autodiscover.example.org), or [https://mail.example.org/owa](https://mail.example.org/owa) do not let you log in, proceed with the steps below to correct the "Certificate Binding" in IIS Manager: + +- Open "**Server Manager**" > Tools > "**Internet Information Services (IIS) Manager**" +- Expand the "**Connections**" server tree on the left-hand side of the IIS Manager +- Expand the "**Sites**" folder + - Click on "**Default Web Site**" + - On the right-hand Actions menu, click on "**Bindings...**" + - A table will appear with different endpoints on the Exchange server > What you are looking for is an entry that looks like the following: + - **Type**: https + - **Host Name**: autodiscover.example.org + - **Port**: 443 + - Double-click on the row, or click one then click the "**Edit**" button to open the settings for that endpoint + - Under "**SSL Certificate**" > Make sure the certificate name matches the following format: "**\[Manual\] autodiscover.example.org @ YYYY/MM/DD**" + - If it does not match the above, use the dropdown menu to correct it and click the "**OK**" button + - **Type**: https + - **Host Name**: mail.example.org + - **Port**: 443 + - Repeat the steps seen above, except this time for "**mail.example.org**" + - Click on "**Exchange Back End**" + - On the right-hand Actions menu, click on "**Bindings...**" + - A table will appear with different endpoints on the Exchange server > What you are looking for is an entry that looks like the following: + - **Type**: https + - **Host Name**: + - **Port**: 444 + - Repeat the steps seen above, ensuring that the "**\[Manual\] autodiscover.example.org @ YYYY/MM/DD**" certificate is selected and applied + - Click the "**OK**" button + - On the left-hand menu under "**Connections**" in IIS Manager, click on the server name itself + - (e.g. "**EXAMPLE-EXCHANGE (DOMAIN\\dptadmin**") + - On the right-hand "**Actions**" menu > Under "Manage Server" > Select "Restart" + - Wait for the IIS server to restart itself, then try accessing the webpages for Exchange that were exhibiting issues logging in + +### Additional Documentation +- [https://www.alitajran.com/install-free-lets-encrypt-certificate-in-exchange-server/](https://www.alitajran.com/install-free-lets-encrypt-certificate-in-exchange-server/) + +## Related Documentation +- [Related Email Documentation](<../../../../reference/Applications/Email/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/email/Proxmox Mail Gateway/Deploying PMG.md b/deployments/Applications/Email/Proxmox Mail Gateway/Integrate PMG with Mailcow.md similarity index 50% rename from deployments/services/email/Proxmox Mail Gateway/Deploying PMG.md rename to deployments/Applications/Email/Proxmox Mail Gateway/Integrate PMG with Mailcow.md index c2007e4..3dfbf78 100644 --- a/deployments/services/email/Proxmox Mail Gateway/Deploying PMG.md +++ b/deployments/Applications/Email/Proxmox Mail Gateway/Integrate PMG with Mailcow.md @@ -1,18 +1,13 @@ --- tags: -- Proxmox Mail Gateway -- PMG -- Mailcow -- Email -- SMTP -- Gateway -- Spam Filtering + - Proxmox Mail Gateway + - PMG + - Mailcow + - Email + - SMTP --- -# Proxmox Mail Gateway Inbound Filtering for Mailcow - ## Purpose - This document defines the procedure for placing `Proxmox Mail Gateway` in front of an existing `Mailcow` server for inbound SMTP filtering. PMG will handle inbound SMTP inspection before delivering accepted mail to Mailcow. @@ -21,32 +16,31 @@ This document covers **inbound SMTP filtering only**. It does not move: -* Outbound SMTP delivery -* DKIM signing -* SMTP submission -* IMAP -* POP3 -* ManageSieve -* Mailcow certificates -* Mailcow web access -* Roundcube access +- Outbound SMTP delivery +- DKIM signing +- SMTP submission +- IMAP +- POP3 +- ManageSieve +- Mailcow certificates +- Mailcow web access +- Roundcube access ## Assumptions - Mailcow is already deployed and functional. Mailcow already handles: -* Mailbox hosting -* User authentication -* Webmail -* Mailcow admin interface -* IMAP -* POP3 -* SMTP submission -* Outbound delivery -* DKIM signing -* TLS certificates for `mail.bunny-lab.io` +- Mailbox hosting +- User authentication +- Webmail +- Mailcow admin interface +- IMAP +- POP3 +- SMTP submission +- Outbound delivery +- DKIM signing +- TLS certificates for `mail.bunny-lab.io` Example environment: @@ -62,14 +56,12 @@ Reverse Proxy: Traefik !!! warning "Inbound SMTP Only" Only move public inbound SMTP port `25` to PMG during this stage. -``` +```text Do not move mail client ports, outbound relay behavior, DKIM signing, or Mailcow web access. ``` ## Architecture - ### Existing Mail Flow - ```text Internet | @@ -81,7 +73,6 @@ Mailcow 192.168.3.61:25 ``` ### Target Mail Flow - ```text Internet | @@ -96,7 +87,6 @@ Mailcow 192.168.3.61:25 ``` ### Final Service Ownership - ```text PMG - Inbound SMTP on port 25 @@ -125,7 +115,6 @@ Traefik ``` ## DNS - Public DNS remains unchanged. ```text @@ -144,12 +133,11 @@ pfSense WAN :25 -> PMG 192.168.3.15:25 !!! note "DNS Does Not Point to PMG Directly" The public MX and A records do not point to the internal PMG IP. -``` +```text NAT controls the internal SMTP destination. ``` ## Firewall and NAT Design - Only public inbound SMTP changes. Change this: @@ -186,12 +174,11 @@ WAN :443 -> Traefik :443 !!! warning "Do Not Move Mail Client Ports to PMG" PMG is an SMTP gateway. -``` +```text Do not forward IMAP, POP3, SMTPS, Submission, or ManageSieve ports to PMG. ``` ## Initial PMG Access - Access the PMG management interface. ```text @@ -204,7 +191,6 @@ Use the `root` credentials configured during PMG installation. Browser certificate warnings are expected when accessing PMG by IP address unless a trusted certificate has already been configured for the management interface. ## Pre-Cutover Connectivity Checks - Confirm PMG can reach Mailcow on SMTP port `25`. Run from the PMG shell: @@ -251,12 +237,11 @@ quit !!! note "Mailcow SMTP Banner" Mailcow commonly identifies its SMTP service as `Postcow`. -``` +```text That is expected. ``` ## PMG Mail Proxy Ports - In PMG, navigate to: ```text @@ -274,12 +259,11 @@ No outbound filtering is configured during this stage. !!! note "Internal SMTP Port" PMG also has an internal SMTP port used for outbound filtering from an internal mail server. -``` +```text This deployment does not use outbound PMG filtering yet. ``` ## PMG Relay Domains - In PMG, navigate to: ```text @@ -298,7 +282,6 @@ This authorizes PMG to accept mail for the domain. If the domain is missing from Relay Domains, PMG may reject inbound mail because it is not configured as responsible for that domain. ## PMG Default Relay - In PMG, navigate to: ```text @@ -327,19 +310,18 @@ Mailcow 192.168.3.61:25 !!! note "Disable MX Lookup" PMG should deliver accepted inbound mail directly to the internal Mailcow server. -``` +```text It should not perform public MX lookup for the local mail domain. ``` !!! note "No Smarthost" Leave `Smarthost` unset or set to `none` for inbound-only filtering. -``` +```text Smarthost configuration is used for outbound relay behavior. ``` ## Mailcow Forwarding Host - Configure Mailcow to trust PMG as a forwarding host. In Mailcow, navigate to: @@ -363,19 +345,18 @@ Inactive !!! note "Forwarding Host Behavior" After cutover, Mailcow sees PMG as the immediate SMTP source for inbound mail. -``` +```text Trusting PMG allows Mailcow to interpret forwarded mail correctly. ``` !!! note "Spam Filtering Placement" PMG is the primary inbound spam and virus filtering system. -``` +```text Leave Mailcow forwarding-host spam filtering inactive to avoid double-filtering messages already inspected by PMG. ``` ## Outbound Mail - Leave outbound mail unchanged. ```text @@ -400,12 +381,11 @@ DMARC record !!! warning "Do Not Move DKIM" DKIM signing applies to outbound mail. -``` +```text This document only moves inbound SMTP filtering. ``` ## Filtering Policy - Initial filtering ownership: ```text @@ -418,12 +398,11 @@ Avoid configuring both PMG and Mailcow to aggressively quarantine the same inbou !!! note "Keep Filtering Boring" PMG should own edge filtering first. -``` +```text Mailcow should continue owning mailbox and client access behavior. ``` ## SMTP NAT Cutover - After PMG relay domains, PMG default relay, and Mailcow forwarding host settings are configured, update the pfSense NAT rule. Change: @@ -460,14 +439,12 @@ Do not change the Traefik web path. !!! warning "Cutover Point" Changing `WAN :25` is the actual inbound mail cutover. -``` +```text External SMTP servers will begin connecting to PMG instead of Mailcow directly. ``` ## Validation - ### External SMTP Reachability - From an external system: ```sh @@ -492,7 +469,7 @@ SMTP banner returned by gateway !!! note "Internal Testing Limitations" Internal tests may not represent public mail flow if NAT reflection or split-horizon DNS is involved. -``` +```text Prefer external testing. ``` @@ -509,7 +486,6 @@ Work mailbox hosted outside Mailcow ``` ### Inbound Delivery - Send an external message to a Mailcow-hosted mailbox. Expected path: @@ -555,7 +531,6 @@ System > Logs or review the relevant Mailcow Postfix and Dovecot logs. ### Mail Client Access - Confirm normal mail client behavior remains unchanged. Test: @@ -581,7 +556,6 @@ Web: 443 -> Traefik -> Mailcow Confirm outbound mail still works by replying from a Mailcow-hosted mailbox to the external sender. ### PMG Queues - Check PMG queues after test delivery. ```text @@ -597,42 +571,39 @@ Queue empty or near-empty after delivery Queue status confirms PMG is not silently holding or deferring mail because of relay, DNS, or delivery errors. ## Validation Checklist - -* [ ] Public MX record points to `mail.bunny-lab.io` -* [ ] `mail.bunny-lab.io` resolves to the correct public WAN IP -* [ ] DNS records are unchanged -* [ ] PMG can reach Mailcow on `192.168.3.61:25` -* [ ] Mailcow SMTP banner is visible from PMG -* [ ] PMG external SMTP port is `25` -* [ ] PMG has `bunny-lab.io` configured as a relay domain -* [ ] PMG default relay points to `192.168.3.61` -* [ ] PMG relay port is `25` -* [ ] PMG relay protocol is `smtp` -* [ ] PMG internal delivery has MX lookup disabled -* [ ] PMG smarthost is unset or `none` -* [ ] Mailcow trusts `192.168.3.15` as a forwarding host -* [ ] Mailcow forwarding-host spam filter is `Inactive` -* [ ] Firewall forwards `WAN :25` to `192.168.3.15:25` -* [ ] Firewall still forwards mail client ports directly to Mailcow -* [ ] Traefik still handles Mailcow / Roundcube web traffic -* [ ] Inbound test mail appears in PMG Tracking Center -* [ ] PMG Tracking Center shows `accepted/delivered` -* [ ] PMG log shows delivery to `192.168.3.61:25` -* [ ] Inbound test mail is delivered to the Mailcow mailbox -* [ ] PMG queue is empty after delivery -* [ ] Mobile email client still works -* [ ] Desktop email client still works -* [ ] Webmail still works -* [ ] Replying outbound from Mailcow still works -* [ ] DKIM behavior is unchanged -* [ ] SPF record is unchanged -* [ ] DMARC record is unchanged -* [ ] Outbound mail routing is unchanged +- [ ] Public MX record points to `mail.bunny-lab.io` +- [ ] `mail.bunny-lab.io` resolves to the correct public WAN IP +- [ ] DNS records are unchanged +- [ ] PMG can reach Mailcow on `192.168.3.61:25` +- [ ] Mailcow SMTP banner is visible from PMG +- [ ] PMG external SMTP port is `25` +- [ ] PMG has `bunny-lab.io` configured as a relay domain +- [ ] PMG default relay points to `192.168.3.61` +- [ ] PMG relay port is `25` +- [ ] PMG relay protocol is `smtp` +- [ ] PMG internal delivery has MX lookup disabled +- [ ] PMG smarthost is unset or `none` +- [ ] Mailcow trusts `192.168.3.15` as a forwarding host +- [ ] Mailcow forwarding-host spam filter is `Inactive` +- [ ] Firewall forwards `WAN :25` to `192.168.3.15:25` +- [ ] Firewall still forwards mail client ports directly to Mailcow +- [ ] Traefik still handles Mailcow / Roundcube web traffic +- [ ] Inbound test mail appears in PMG Tracking Center +- [ ] PMG Tracking Center shows `accepted/delivered` +- [ ] PMG log shows delivery to `192.168.3.61:25` +- [ ] Inbound test mail is delivered to the Mailcow mailbox +- [ ] PMG queue is empty after delivery +- [ ] Mobile email client still works +- [ ] Desktop email client still works +- [ ] Webmail still works +- [ ] Replying outbound from Mailcow still works +- [ ] DKIM behavior is unchanged +- [ ] SPF record is unchanged +- [ ] DMARC record is unchanged +- [ ] Outbound mail routing is unchanged ## Troubleshooting - ### Inbound Mail Never Reaches PMG - Verify NAT. ```text @@ -655,7 +626,6 @@ PMG > Tracking Center ``` ### PMG Receives Mail but Does Not Deliver to Mailcow - Verify PMG relay settings. ```text @@ -687,7 +657,6 @@ Expected banner: ``` ### PMG Shows Reverse DNS Warning for Mailcow - A warning like this is not automatically a failure: ```text @@ -697,7 +666,6 @@ inverse host lookup failed: Unknown host If the connection still reports port `25` as open, SMTP connectivity is working. ### Mailcow Rejects Mail from PMG - Verify Mailcow trusts PMG as a forwarding host. ```text @@ -713,7 +681,6 @@ bunny-lab.io Verify the recipient mailbox or alias exists in Mailcow. ### Mail Clients Stop Working - Verify only inbound SMTP port `25` was moved to PMG. These ports should still forward directly to Mailcow: @@ -736,7 +703,6 @@ Mailcow -> client access and mailbox services ``` ### Roundcube or Mailcow Web UI Stops Working - Verify web traffic was not moved to PMG. Expected path: @@ -749,7 +715,6 @@ WAN :443 -> Traefik :443 PMG should not replace Traefik for Mailcow or Roundcube web access. ### Outbound Mail Stops Working - Outbound mail should not change during this deployment. Verify no changes were made to: @@ -764,7 +729,6 @@ Public DNS records ``` ### Spam Filtering Behavior Is Confusing - Use one primary inbound filtering authority. Recommended initial state: @@ -777,7 +741,6 @@ Mailcow = mailbox hosting and client access Avoid dual aggressive quarantine policies until basic mail flow is stable. ## Confirmed Final State - After Stage 1, the environment should operate as follows: ```text @@ -818,7 +781,6 @@ Mailcow / Roundcube web access ``` ## Deployment Status - This document completes Stage 1 of the PMG deployment. ```text @@ -833,320 +795,8 @@ PMG = inbound SMTP filtering only Mailcow = mailboxes, webmail, authenticated submission, certificates, DKIM, outbound delivery, user-facing mail services ``` -## Enabling Direct Unfiltered Communication Between Mailcow and PMG -This workflow describes how to configure Mailcow to trust Proxmox Mail Gateway (PMG) as an upstream SMTP relay after PMG has been placed in front of Mailcow for inbound filtering. It prevents Mailcow from applying sender-domain validation and duplicate spam filtering to messages that PMG has already accepted, including messages released from the PMG quarantine. - -In the current Bunny Lab design, this trust applies to the inbound delivery path from `192.168.3.15` to `192.168.3.61`. Mailcow continues to send outbound mail directly to the Internet and does not relay outbound mail through PMG. - -!!! info "Assumptions" - - Mailcow is already deployed at `192.168.3.61`. - - PMG is already deployed at `192.168.3.15`. - - Public inbound SMTP port `25` is forwarded to PMG. - - PMG relays accepted inbound mail to Mailcow on `192.168.3.61:25`. - - Mailcow remains responsible for mailbox hosting, authenticated submission, DKIM signing, and outbound delivery. - - You have `root` access to the Mailcow host. - -!!! warning "Trust Only the PMG Host" - Add only `192.168.3.15/32` to Mailcow's trusted networks. Do not trust the entire `192.168.3.0/24` subnet unless every system on that subnet is authorized to relay mail through Mailcow without authentication. - -## Architecture -```text -Internet - | - v -pfSense WAN :25 - | - v -PMG 192.168.3.15:25 - | - v -Mailcow 192.168.3.61:25 - | - v -User Mailbox -``` - -PMG is the primary inbound spam and virus filtering authority. Mailcow should accept the SMTP session from PMG as a trusted relay and avoid repeating edge filtering or rejecting PMG-generated envelope senders. - -## Symptoms -Use this workflow when PMG receives and processes mail correctly, but Mailcow rejects or defers the final delivery. - -Common symptoms include: - -- A released PMG quarantine message remains in the PMG deferred queue -- Clicking **Flush** in PMG does not deliver the message -- PMG repeatedly attempts delivery to `192.168.3.61:25` -- Mailcow rejects a PMG-generated envelope sender such as `postmaster@lab-mail-gw-01.bunny-lab.io` -- PMG reports an SMTP response similar to: - -```text -450 4.1.8 Sender address rejected: Domain not found -``` - -The visible message sender may be valid even when Mailcow rejects the PMG-generated envelope sender used during quarantine release. - -## Confirm the PMG Source Address -Mailcow must trust the source address it actually sees on the SMTP connection. Confirm that address before changing the configuration. - -On the Mailcow host, run: - -```sh -cd /opt/mailcow-dockerized -docker compose logs --since 5m postfix-mailcow -``` - -Flush or resend a test message from PMG, then locate the connection line. - -Expected connection source: - -```text -connect from lab-mail-gw-01[192.168.3.15] -``` - -If Mailcow sees a different address because of NAT, a load balancer, or another SMTP proxy, use that observed address instead of `192.168.3.15`. - -## Configure Mailcow Forwarding Hosts -Configure PMG as a trusted forwarding host in the Mailcow WebUI. - -- Navigate to "**System > Configuration > Options > Forwarding Hosts**" - - Add `192.168.3.15` - - Set the forwarding-host spam filter option to **Inactive** - - Save the configuration - -!!! note "Forwarding Host Behavior" - The forwarding-host entry tells Mailcow that PMG is the immediate upstream SMTP relay. Keeping the forwarding-host spam filter inactive avoids applying a second aggressive spam-filtering layer to mail that PMG has already inspected. - -## Configure Postfix Trusted Networks -The forwarding-host entry does not replace Postfix `mynetworks`. Add PMG to `mynetworks` so Postfix treats SMTP sessions from PMG as trusted and evaluates `permit_mynetworks` before sender-domain restrictions. - -### Inspect the Existing Trusted Networks -Before modifying `mynetworks`, determine the currently active value. Mailcow automatically populates `mynetworks` with its loopback and Docker networks. Defining your own value replaces that automatically generated configuration, so you must preserve the existing entries. - -On the Mailcow host, run: - -```sh -cd /opt/mailcow-dockerized -docker compose exec postfix-mailcow postconf mynetworks -``` - -Example output: - -```text -mynetworks = 127.0.0.0/8 172.22.1.0/24 [::1]/128 -``` - -At this point, append the PMG address as a single-host CIDR. Do not remove any existing networks. - -Expected result: - -```text -mynetworks = 127.0.0.0/8 172.22.1.0/24 [::1]/128 192.168.3.15/32 -``` - -### Update the Persistent Postfix Override - -Edit the Postfix override file: - -```sh -nano /opt/mailcow-dockerized/data/conf/postfix/extra.cf -``` - -If `mynetworks` config line already exists, append the PMG address while preserving the existing values. If the key does not exist, create it using the value discovered in the previous step. - -```ini title="/opt/mailcow-dockerized/data/conf/postfix/extra.cf" -mynetworks = 127.0.0.0/8 172.22.1.0/24 [::1]/128 192.168.3.15/32 -``` - -!!! warning "Do Not Replace Existing Networks" - The `mynetworks` directive replaces Mailcow's automatically generated value. Removing the existing loopback or Docker networks may prevent Postfix from functioning correctly. Always preserve the existing entries and append the PMG address. - -### Restart Mailcow Postfix -Apply the persistent override: - -```sh -cd /opt/mailcow-dockerized -docker compose restart postfix-mailcow -``` - -## Validate the Trusted Relay Configuration -Confirm the active Postfix configuration after the restart: - -```sh -cd /opt/mailcow-dockerized -docker compose exec postfix-mailcow postconf mynetworks -``` - -Expected result: - -```text -mynetworks = 127.0.0.0/8 172.22.1.0/24 [::1]/128 192.168.3.15/32 -``` - -Confirm the active SMTP restriction order includes `permit_mynetworks`: - -```sh -cd /opt/mailcow-dockerized -docker compose exec postfix-mailcow postconf \ - smtpd_sender_restrictions \ - smtpd_recipient_restrictions \ - smtpd_relay_restrictions -``` - -The exact restriction lists may change between Mailcow releases. Confirm that `permit_mynetworks` remains present and that the effective configuration recognizes `192.168.3.15/32` as trusted. - -## Retry Deferred PMG Mail -After Mailcow trusts PMG, retry the deferred message from the PMG WebUI. - -- Navigate to "**Administration > Queue Administration > Deferred Mail**" - - Select the deferred message - - Click **Flush** - -Alternatively, run the following on PMG: - -```sh -postqueue -f -``` - -To retry one specific queue item: - -```sh -postsuper -r -postqueue -f -``` - -## Validate Mail Delivery -Monitor Mailcow while PMG retries the message: - -```sh -cd /opt/mailcow-dockerized -docker compose logs -f postfix-mailcow -``` - -Mailcow should accept the SMTP transaction and return a successful queue response similar to: - -```text -250 2.0.0 Ok: queued as -``` - -On PMG, confirm the deferred queue no longer contains the message: - -```sh -postqueue -p -``` - -Confirm the message is present in the destination Mailcow mailbox. - -!!! success "Trusted PMG Delivery Confirmed" - The workflow is complete when Mailcow accepts mail from `192.168.3.15`, the PMG deferred queue clears, and the destination mailbox receives the message. - -## Security Boundaries -Adding `192.168.3.15/32` to `mynetworks` permits PMG to relay mail through Mailcow without SMTP authentication. This is appropriate only while PMG remains a controlled gateway host. - -Maintain the following boundaries: - -- Restrict the trusted entry to `192.168.3.15/32` -- Prevent other hosts from impersonating the PMG source address -- Keep Mailcow port `25` restricted to expected SMTP sources where practical -- Confirm PMG is not configured as an unrestricted open relay -- Continue routing authenticated client submission directly to Mailcow on ports `465` and `587` -- Continue routing public inbound SMTP port `25` to PMG rather than directly to Mailcow - -## Troubleshooting -### Mailcow Still Returns `450 4.1.8` -Confirm Mailcow is using the updated configuration: - -```sh -cd /opt/mailcow-dockerized -docker compose exec postfix-mailcow postconf mynetworks -``` - -If `192.168.3.15/32` is missing, verify `/opt/mailcow-dockerized/data/conf/postfix/extra.cf` and restart `postfix-mailcow`. - -### Mailcow Sees a Different Source Address -Inspect the Mailcow Postfix logs: - -```sh -cd /opt/mailcow-dockerized -docker compose logs --since 5m postfix-mailcow -``` - -Use the IP address shown in the `connect from ...[IP_ADDRESS]` line. Do not assume Mailcow sees the PMG management address when NAT or an SMTP proxy exists between the systems. - -### PMG Flushes the Message but the Mailbox Does Not Receive It -Determine whether Mailcow accepted the message. - -If Mailcow returned `250 2.0.0`, the PMG-to-Mailcow relay succeeded. Continue troubleshooting inside Mailcow by reviewing Postfix, Rspamd, and Dovecot logs. - -```sh -cd /opt/mailcow-dockerized -docker compose logs --since 10m postfix-mailcow rspamd-mailcow dovecot-mailcow -``` - -If Mailcow returned a `4xx` or `5xx` response, use the complete SMTP response as the controlling error and resolve that policy or recipient failure before retrying again. - -### Mailcow Applies Spam Filtering Again -Confirm the PMG entry under "**System > Configuration > Options > Forwarding Hosts**" has its spam filter set to **Inactive**. - -PMG should remain the primary inbound filtering authority in this deployment. - -### Mail Clients or Outbound Delivery Stop Working -This workflow does not change client access or outbound delivery. - -Confirm the existing service ownership remains: - -```text -Inbound SMTP: Internet -> PMG -> Mailcow -Outbound SMTP: Mailcow -> Internet -SMTP Submission: Clients -> Mailcow -IMAP and POP3: Clients -> Mailcow -Webmail and Admin: Internet -> Traefik -> Mailcow -``` - -Do not redirect ports `465`, `587`, `993`, `995`, `110`, `143`, or `4190` to PMG. - -## Rollback -Remove the PMG forwarding-host entry from the Mailcow WebUI only when PMG is no longer the upstream SMTP gateway. - -Then edit: - -```sh -nano /opt/mailcow-dockerized/data/conf/postfix/extra.cf -``` - -Restore the previous `mynetworks` value: - -```ini title="/opt/mailcow-dockerized/data/conf/postfix/extra.cf" -mynetworks = 127.0.0.0/8 172.22.1.0/24 [::1]/128 -``` - -Restart Postfix: - -```sh -cd /opt/mailcow-dockerized -docker compose restart postfix-mailcow -``` - -!!! warning "Coordinate the SMTP Path Before Rollback" - Do not remove PMG trust while public inbound SMTP still routes through PMG. Mailcow may begin rejecting legitimate messages relayed from PMG. - -## Confirmed Final State -The validated Bunny Lab configuration is: - -```text -PMG Address: 192.168.3.15 -Mailcow Address: 192.168.3.61 -Mailcow Forwarder: 192.168.3.15 -Forwarder Spam Check: Inactive -Postfix mynetworks: 127.0.0.0/8 172.22.1.0/24 [::1]/128 192.168.3.15/32 -``` - -The resulting mail flow is: - -```text -Inbound SMTP: -Internet -> pfSense WAN :25 -> PMG 192.168.3.15:25 -> Mailcow 192.168.3.61:25 - -Outbound SMTP: -Mailcow -> Internet -``` +## Maintain Mail Delivery +For sender-validation and trusted-relay failures after integration, follow [Repair Trusted Mail Delivery Between PMG and Mailcow](<../../../../workflows/Applications/Email/Proxmox Mail Gateway/Repair Trusted Mail Delivery Between PMG and Mailcow.md>). +## Related Documentation +- [Related Email Documentation](<../../../../reference/Applications/Email/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/Applications/Email/cPanel/Create a cPanel Email Server.md b/deployments/Applications/Email/cPanel/Create a cPanel Email Server.md new file mode 100644 index 0000000..c089de3 --- /dev/null +++ b/deployments/Applications/Email/cPanel/Create a cPanel Email Server.md @@ -0,0 +1,17 @@ +--- +tags: + - cPanel + - Email +--- + +## Purpose +This documentation helps you deploy an email server within a cPanel hosted environment. + +!!! warning "Incomplete Procedure" + The deployment steps remain a scaffold. No completed cPanel mail-server procedure is recorded here. + +!!! note "Assumptions" + It is assumed that the cPanel environment is set up (prior) to following this documentation, as deploying cPanel itself is not covered in this document. + +## Related Documentation +- [Related Email Documentation](<../../../../reference/Applications/Email/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/email/iRedMail/Deploy iRedMail.md b/deployments/Applications/Email/iRedMail/Deploy iRedMail.md similarity index 94% rename from deployments/services/email/iRedMail/Deploy iRedMail.md rename to deployments/Applications/Email/iRedMail/Deploy iRedMail.md index 07701ac..8f3e995 100644 --- a/deployments/services/email/iRedMail/Deploy iRedMail.md +++ b/deployments/Applications/Email/iRedMail/Deploy iRedMail.md @@ -4,7 +4,7 @@ tags: - Email --- -**Purpose**: +## Purpose Self-Hosted Open-Source email server that can be setup in minutes, and is enterprise-grade if upgraded with an iRedAdmin-Pro license. !!! note "Assumptions" @@ -20,8 +20,9 @@ The instructions below are specific to my homelab environment, but can be easily Ensure the FQDN of the server is correctly set in `/etc/hostname`. The `/etc/hosts` file will be automatically injected using the FQDN from `/etc/hostname` in a script further down, don't worry about editing it. ## Disable SELinux -iRedMail doesn't work with SELinux, so please disable it by setting below value in its config file /etc/selinux/config. After server reboot, SELinux will be completely disabled. -``` sh +iRedMail doesn't work with SELinux, so please disable it by setting below value in its config file /etc/selinux/config. After server reboot, SELinux will be completely disabled. + +```sh # Elevate to Root User sudo su @@ -33,11 +34,11 @@ setenforce 0 1. If you prefer to let SELinux prints warnings instead of enforcing, you can set this value instead: `SELINUX=permissive` ## iRedMail Installation - ### Set Domain and iRedMail Version Start by connecting to the server / VM via SSH, then set silent deployment variables below. -``` sh -# Define some deployment variables. + +```sh +# Define some deployment variables. VERSION="1.6.8" # (1) MAIL_DOMAIN="bunny-lab.io" # (2) ``` @@ -51,7 +52,7 @@ You will then proceed to bootstrap a silent unattended installation of iRedMail. You absolutely need to ensure that `/var/vmail` has a lot of space. At least 16GB. This is where all of your emails / mailboxes / a lot of settings will be. If possible, create a second physical/virtual disk specifically for the `/var` partition, or specifically for `/var/vmail` at minimum, so you can expand it over time if necessary. LVM-based provisioning is recommended but not required. ### Install iRedMail -``` sh +```sh # Automatically configure the /etc/hosts file to point to the server listed in "/etc/hostname". sudo sed -i "1i 127.0.0.1 $(cat /etc/hostname) $(cut -d '.' -f 1 /etc/hostname) localhost localhost.localdomain localhost4 localhost4.localdomain4" /etc/hosts @@ -86,7 +87,8 @@ AUTO_USE_EXISTING_CONFIG_FILE=y \ ``` When the installation is completed, take note of any output it gives you for future reference. Then reboot the server to finalize the server installation. -``` + +```text reboot ``` @@ -94,11 +96,10 @@ reboot When you deploy iRedMail, it will give you a username and password for the postmaster account. If you accidentally forget to document this, you can log back into the server via SSH and see the credentials at `/root/iRedMail-$VERSION/iRedMail.tips`. This file is critical and contains passwords and DNS information such as DKIM record information as well. ## Networking Configuration - ### Nested Reverse Proxy Configuration In my homelab environment, I run Traefik reverse proxy in front of everything, which includes the NGINX reverse proxy that iRedMail creates. In my scenario, I have to make some custom adjustments to the reverse proxy dynamic configuration data to ensure it will step aside and let the NGINX reverse proxy inside of iRedMail handle everything, including handling its own SSL termination with Let's Encrypt. -``` sh +```sh tcp: routers: mail-tcp-router: @@ -121,12 +122,12 @@ At this point, we want to set up automatic Let's Encrypt SSL termination inside #### Generate SSL Certificate === "Debian/Ubuntu" - ``` sh + ```sh # Download the Certbot sudo apt update sudo apt install -y certbot sudo certbot certonly --webroot -w /var/www/html -d mail.bunny-lab.io - + # Set up Symbolic Links (Where iRedMail Expects Them) sudo mv /etc/ssl/certs/iRedMail.crt{,.bak} sudo mv /etc/ssl/private/iRedMail.key{,.bak} @@ -136,10 +137,10 @@ At this point, we want to set up automatic Let's Encrypt SSL termination inside # Restart iRedMail Services sudo systemctl restart postfix dovecot nginx ``` - + === "CentOS/Rocky/AlmaLinux" - ``` sh + ```sh # Download the Certbot sudo yum install -y epel-release sudo yum install -y certbot @@ -156,15 +157,17 @@ At this point, we want to set up automatic Let's Encrypt SSL termination inside ``` #### Configure Automatic Renewal -To automate the renewal process, set up a cron job that runs the certbot renew command regularly. This command will renew certificates that are due to expire within 30 days. +To automate the renewal process, set up a cron job that runs the certbot renew command regularly. This command will renew certificates that are due to expire within 30 days. Open the crontab editor with the following command: -``` + +```sh sudo crontab -e ``` Add the following line to run the renewal process daily at 3:01 AM: -``` + +```text 1 3 * * * certbot renew --post-hook 'systemctl restart postfix dovecot nginx' ``` @@ -191,7 +194,7 @@ Lastly, we need to set up port forwarding to open the ports necessary for the se ## Install iRedAdmin-Pro When it comes to adding extra features, start by copying the data from this [Bunny Lab repository](https://git.bunny-lab.io/bunny-lab/iRedAdmin-Pro-SQL) to the following folder by running these commands first: -``` sh +```sh # Stop the iRedMail Services sudo systemctl stop postfix dovecot nginx @@ -212,14 +215,14 @@ At this point, if you want to use iRedAdmin-Pro, you either have a valid license There is someone else who outlined all of these changes, and additional (aesthetic) ones, like removing the renew license button from the license page, but the core functionality is seen below. If you want to see the original repository this was inspired from, it can be found [Here](https://github.com/marcus-alicia/iRedAdmin-Pro-SQL) -``` sh +```sh # Take permission of the python script sudo chown nicole:nicole /opt/www/iRedAdmin-2.5/libs/sysinfo.py ``` === "Original Activation Function" - ```jsx title="/opt/www/iRedAdmin-2.5/libs/sysinfo.py" + ```python title="/opt/www/iRedAdmin-2.5/libs/sysinfo.py" def get_license_info(): if len(__id__) != 32: web.conn_iredadmin.delete("updatelog") @@ -248,10 +251,10 @@ sudo chown nicole:nicole /opt/www/iRedAdmin-2.5/libs/sysinfo.py except Exception as e: return False, web.urlquote(e) ``` - + === "Bypassed Activation Function" - ```jsx title="/opt/www/iRedAdmin-2.5/libs/sysinfo.py" + ```python title="/opt/www/iRedAdmin-2.5/libs/sysinfo.py" def get_license_info(): return True, { "status": "active", @@ -267,7 +270,7 @@ sudo chown nicole:nicole /opt/www/iRedAdmin-2.5/libs/sysinfo.py } ``` -``` sh +```sh # Revert permission of the python script sudo chown iredadmin:iredadmin /opt/www/iRedAdmin-2.5/libs/sysinfo.py @@ -276,4 +279,7 @@ sudo reboot ``` !!! success "Successful Activation" - At this point, if you navigate to the [iRedAdmin-Pro License Page](https://mail.bunny-lab.io/iredadmin/system/license) you should see the server is activated successfully. \ No newline at end of file + At this point, if you navigate to the [iRedAdmin-Pro License Page](https://mail.bunny-lab.io/iredadmin/system/license) you should see the server is activated successfully. + +## Related Documentation +- [Related Email Documentation](<../../../../reference/Applications/Email/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/email/mailcow.md b/deployments/Applications/Email/mailcow.md similarity index 78% rename from deployments/services/email/mailcow.md rename to deployments/Applications/Email/mailcow.md index 9a892c6..30222bb 100644 --- a/deployments/services/email/mailcow.md +++ b/deployments/Applications/Email/mailcow.md @@ -1,10 +1,8 @@ --- - tags: -- Mailcow -- Email -- Docker - + - Mailcow + - Email + - Docker --- ## Purpose @@ -23,10 +21,10 @@ apt install docker-compose-plugin # (3) systemctl enable --now docker # (4) ``` -1. Make yourself root. -2. Install `Docker` -3. Install `Docker-Compose` -4. Make docker run automatically when the server is booted. +1. Make yourself root. +2. Install `Docker` +3. Install `Docker-Compose` +4. Make docker run automatically when the server is booted. ### Download and Deploy Mailcow Run the following commands to pull down the mailcow deployment files and install them with docker. Go get a cup of coffee as the `docker compose pull` command may take a while to run. @@ -43,8 +41,8 @@ docker-compose pull # (2) docker-compose up -d ``` -1. Generate a configuration file. Use a FQDN (`host.domain.tld`) as hostname when asked. -2. If you get an error about the ports of the `nginx-mailcow` service in the `docker-compose.yml` stack, change the ports for that service as follows: +1. Generate a configuration file. Use a FQDN (`host.domain.tld`) as hostname when asked. +2. If you get an error about the ports of the `nginx-mailcow` service in the `docker-compose.yml` stack, change the ports for that service as follows: ```yaml ports: @@ -69,10 +67,10 @@ WAN :143 -> Mailcow :143 WAN :4190 -> Mailcow :4190 ``` -Mail protocol ports should be sent directly to the Mailcow server. Traefik should not terminate or proxy the SMTP, SMTPS, Submission, IMAP, IMAPS, POP3, POP3S, or ManageSieve ports. +Mail protocol ports should be sent directly to the Mailcow server. Traefik should not terminate or proxy the SMTP, SMTPS, Submission, IMAP, IMAPS, POP3, POP3S, or ManageSieve ports. ### Reverse-Proxy Configuration -For the purposes of this document, it will be assumed that you are deploying Mailcow behind Traefik for web traffic only. Traefik should pass HTTPS through transparently, allowing Mailcow to manage and serve its own certificates. +For the purposes of this document, it will be assumed that you are deploying Mailcow behind Traefik for web traffic only. Traefik should pass HTTPS through transparently, allowing Mailcow to manage and serve its own certificates. You can use the following dynamic configuration file to achieve this: @@ -120,7 +118,7 @@ tcp: ``` ### Traefik-Specific Configuration -Traefik only needs the standard HTTP and HTTPS entrypoints for Mailcow web traffic. Mail protocol ports should not be exposed through Traefik if the firewall is forwarding those ports directly to Mailcow. +Traefik only needs the standard HTTP and HTTPS entrypoints for Mailcow web traffic. Mail protocol ports should not be exposed through Traefik if the firewall is forwarding those ports directly to Mailcow. ```yaml #Entrypoints @@ -156,11 +154,15 @@ docker compose restart postfix-mailcow dovecot-mailcow nginx-mailcow ### Login to Mailcow At this point, the Mailcow server has been deployed so you can log into it. -* **Administrators**: `https://${MAILCOW_HOSTNAME}/admin` (Username: `admin` | Password: `moohoo`) -* **Regular Mailbox Users**: `https://${MAILCOW_HOSTNAME}` (*FQDN only*) +- **Administrators**: `https://${MAILCOW_HOSTNAME}/admin` (Username: `admin` | Password: `moohoo`) +- **Regular Mailbox Users**: `https://${MAILCOW_HOSTNAME}` (*FQDN only*) ### Mail-Client Considerations You need to ensure that you generate an app password if you have MFA enabled within Mailcow. (MFA is non-functional in Roundcube/SoGo, you set it up via Mailcow itself). You can access it via the Mailcow configuration page: https://mail.bunny-lab.io/user, then look for the "**App Passwords**" tab. ### Running Updates If you want to run updates, just SSH into the server, and navigate to `/opt/mailcow-dockerized` and run `./update.sh`. I recommend avoiding the IPv6 implementation section. Be patient, and the upgrade will be fully-automated. + +## Related Documentation +- [Related Email Documentation](<../../../reference/Applications/Email/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/productivity/Collabora Code Server.md b/deployments/Applications/Files and Collaboration/Collabora Code Server.md similarity index 91% rename from deployments/services/productivity/Collabora Code Server.md rename to deployments/Applications/Files and Collaboration/Collabora Code Server.md index 08669ac..4a2952c 100644 --- a/deployments/services/productivity/Collabora Code Server.md +++ b/deployments/Applications/Files and Collaboration/Collabora Code Server.md @@ -5,17 +5,17 @@ tags: - Docker --- -## Purpose: -The Collabora CODE Server is used by Nextcloud Office to open and edit documents and spreadsheets collaboratively. When Nextcloud is not deployed in a [Nextcloud AIO](./nextcloud-aio.md) way, and is instead installed not as a container, you (may) run into stability issues with Collabora CODE Server just randomly breaking and not allowing users to edit documents. If this happens, you can follow this document to stand-up a dedicated Collabora CODE Server on the same host as your Nextcloud server. +## Purpose +The Collabora CODE Server is used by Nextcloud Office to open and edit documents and spreadsheets collaboratively. When Nextcloud is not deployed in a [Nextcloud AIO]() way, and is instead installed not as a container, you (may) run into stability issues with Collabora CODE Server just randomly breaking and not allowing users to edit documents. If this happens, you can follow this document to stand-up a dedicated Collabora CODE Server on the same host as your Nextcloud server. !!! info "Assumptions" - - - It is assumed that you are running an ACME Certificate Bot on your Nextcloud server to generate certificates for Nextcloud. - - It is also assumed that you are running Ubuntu Server 24.04.3 LTS. *This document does not outline the process for setting up an ACME Certificate Bot*. + + - It is assumed that you are running an ACME Certificate Bot on your Nextcloud server to generate certificates for Nextcloud. + - It is also assumed that you are running Ubuntu Server 24.04.3 LTS. *This document does not outline the process for setting up an ACME Certificate Bot*. - It is lastly assumed that (until changes are made to allow such) this will only work for internal access. Unless you port-forward port `9980` Collabora will not function for public internet-facing access. ### Install Docker and Configure Portainer -The first thing you need to do is install Docker then Portainer. You can do this by following the [Portainer Deployment](../../platforms/containerization/docker/deploy-portainer.md) documentation. +The first thing you need to do is install Docker then Portainer. You can do this by following the [Portainer Deployment](<../../Containers/Docker/Deploy Portainer.md>) documentation. ### Portainer Stack ```yaml title="docker-compose.yml" @@ -67,6 +67,7 @@ CODESERVER_ADMIN_PASSWORD=ChangeThisPassword ## NGINX Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml title="/opt/collabora/nginx.conf" map $http_upgrade $connection_upgrade { default upgrade; @@ -116,7 +117,7 @@ Now that the Collabora CODE Server was deployed and instructed to use the existi - Navigate to "**Administration Settings**" - In the left-hand "**Administration**" sidebar, look for something like "**Office**" or "**Nextcloud Office**" and click on it - Check the radio box that says "**Use your own server**" - - For the URL, enter `https://cloud.bunny-lab.io:9980` and uncheck the "**Disable certificate verification (insecure)**" checkbox, then click the "**Save**" button. + - For the URL, enter `https://cloud.bunny-lab.io:9980` and uncheck the "**Disable certificate verification (insecure)**" checkbox, then click the "**Save**" button. !!! success "Collabora Online Server is Reachable" At this point, you should see a green banner at the top of the Nextcloud webpage stating something like "**Collabora Online Development Edition 25.04.7.2 a246f9ab3c**". This would indicate that Nextcloud should be able to successfully talk with the Collabora CODE Server and that you can now proceed to verify that everything is working by trying to create and edit some documents and spreadsheets. @@ -124,3 +125,5 @@ Now that the Collabora CODE Server was deployed and instructed to use the existi ### Administrating Collabora CODE Server As aforementioned, we can manage Collabora CODE Server sessions and useful metrics about who is editing documents and being able to terminate their sessions if they get stuck or something can be useful. You can login to the management web interface at https://cloud.bunny-lab.io:9980/browser/dist/admin/admin.html using the `CODESERVER_ADMIN_USER` and `CODESERVER_ADMIN_PASSWORD` credentials. +## Related Documentation +- [Related Files and Collaboration Documentation](<../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/productivity/Nextcloud AIO.md b/deployments/Applications/Files and Collaboration/Nextcloud AIO.md similarity index 94% rename from deployments/services/productivity/Nextcloud AIO.md rename to deployments/Applications/Files and Collaboration/Nextcloud AIO.md index 997e436..59f6374 100644 --- a/deployments/services/productivity/Nextcloud AIO.md +++ b/deployments/Applications/Files and Collaboration/Nextcloud AIO.md @@ -6,8 +6,8 @@ tags: - Docker --- -**Purpose**: -Deploy a Nextcloud AIO Server. [Official Nextcloud All-in-One Documentation](https://github.com/nextcloud/all-in-one). +## Purpose +Deploy a Nextcloud AIO Server. [Official Nextcloud All-in-One Documentation](https://github.com/nextcloud/all-in-one). This version of Nextcloud consists of 12 containers that are centrally managed by a single "master" container. It is more orchestrated and automates the implementation of Nextcloud Office, Nextcloud Talk, and other integrations / apps. !!! note "Assumptions" @@ -42,7 +42,7 @@ This version of Nextcloud consists of 12 containers that are centrally managed b nextcloud_aio_mastercontainer: name: nextcloud_aio_mastercontainer ``` - + === "Extended Docker-Compose.yml" ```yaml title="docker-compose.yml" @@ -146,7 +146,7 @@ http: https-redirect: redirectscheme: - scheme: https + scheme: https nextcloud-chain: chain: @@ -156,14 +156,19 @@ http: - nextcloud-secure-headers ``` -## Initial Setup +## Initial Setup You will need to navigate to https://192.168.3.29:8080 to access the Nextcloud AIO configuration tool. This is where you will get the AIO password, encryption passphrase for backups, and be able to configure the timezone, among other things. ### Domain Validation -It will ask you to provide a domain name. In this example, we will use `cloud.bunny-lab.io`. Assuming you have configured the Traefik reverse proxy as seen above, when you press the "**Validate Domain**" button, Nextcloud will spin up a container named something similar to `domain-validator`. This will spin up a server listening on https://cloud.bunny-lab.io. If you visit that address, it should give you something similar to `f940935260b41691ac2246ba9e7823a301a1605ae8a023ee`. This will confirm that the domain validation will succeed. +It will ask you to provide a domain name. In this example, we will use `cloud.bunny-lab.io`. Assuming you have configured the Traefik reverse proxy as seen above, when you press the "**Validate Domain**" button, Nextcloud will spin up a container named something similar to `domain-validator`. This will spin up a server listening on https://cloud.bunny-lab.io. If you visit that address, it should give you something similar to `f940935260b41691ac2246ba9e7823a301a1605ae8a023ee`. This will confirm that the domain validation will succeed. !!! warning "Domain Validation Failing" If visiting the web server at https://cloud.bunny-lab.io results in an error 502 or 404, try to destroy the domain validation container in Portainer / Docker, then click the validation button in the Nextcloud AIO WebUI to spin up a new container automatically, at which point it should be function. ### Configuring Additional Packages -At this point, the rest of the setup is fairly straightforward. You just check every checkbox for the apps you want to install automatically, and be patient while Nextcloud deploys about 11 containers. You can track the progress more accurately if you log into Portainer and watch the container listing and logs to follow-along until every container reports "**Healthy**" indicating everything is ready, then press the "**Refresh**" button on the Nextcloud AIO WebUI to confirm it's ready to be used. \ No newline at end of file +At this point, the rest of the setup is fairly straightforward. You just check every checkbox for the apps you want to install automatically, and be patient while Nextcloud deploys about 11 containers. You can track the progress more accurately if you log into Portainer and watch the container listing and logs to follow-along until every container reports "**Healthy**" indicating everything is ready, then press the "**Refresh**" button on the Nextcloud AIO WebUI to confirm it's ready to be used. + +## Related Documentation +- [Collabora Office Integration]() — Review the standalone office-server option and its Nextcloud requirements. +- [Related Files and Collaboration Documentation](<../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/productivity/nextcloud.md b/deployments/Applications/Files and Collaboration/Nextcloud.md similarity index 78% rename from deployments/services/productivity/nextcloud.md rename to deployments/Applications/Files and Collaboration/Nextcloud.md index c07c08c..e5e4346 100644 --- a/deployments/services/productivity/nextcloud.md +++ b/deployments/Applications/Files and Collaboration/Nextcloud.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Deploy a Nextcloud and PostgreSQL database together. +## Purpose +Deploy a Nextcloud and PostgreSQL database together. ```yaml title="docker-compose.yml" version: "2.1" @@ -48,7 +49,7 @@ services: volumes: - /srv/containers/nextcloud/db:/var/lib/postgresql/data ports: - - 5432:5432 + - 5432:5432 restart: always networks: docker_network: @@ -69,3 +70,7 @@ NEXTCLOUD_ADMIN_USER=admin NEXTCLOUD_ADMIN_PASSWORD=SomeSuperSecurePassword NEXTCLOUD_TRUSTED_DOMAINS=cloud.bunny-lab.io ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Files and Collaboration Documentation](<../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/Applications/Files and Collaboration/OnlyOffice EE.md b/deployments/Applications/Files and Collaboration/OnlyOffice EE.md new file mode 100644 index 0000000..eef1fd4 --- /dev/null +++ b/deployments/Applications/Files and Collaboration/OnlyOffice EE.md @@ -0,0 +1,76 @@ +--- +tags: + - OnlyOffice + - Productivity + - Docker +--- + +## Purpose +ONLYOFFICE offers a secure online office suite highly compatible with MS Office formats. Generally used with Nextcloud to edit documents directly within the web browser. + +```yaml title="docker-compose.yml" +version: '3' + +services: + app: + image: onlyoffice/documentserver-ee + ports: + - 80:80 + - 443:443 + volumes: + - /srv/containers/onlyoffice/DocumentServer/logs:/var/log/onlyoffice + - /srv/containers/onlyoffice/DocumentServer/data:/var/www/onlyoffice/Data + - /srv/containers/onlyoffice/DocumentServer/lib:/var/lib/onlyoffice + - /srv/containers/onlyoffice/DocumentServer/db:/var/lib/postgresql + - /srv/containers/onlyoffice/DocumentServer/fonts:/usr/share/fonts/truetype/custom + - /srv/containers/onlyoffice/DocumentServer/forgotten:/var/lib/onlyoffice/documentserver/App_Data/cache/files/forgotten + - /srv/containers/onlyoffice/DocumentServer/rabbitmq:/var/lib/rabbitmq + - /srv/containers/onlyoffice/DocumentServer/redis:/var/lib/redis + labels: + - "traefik.enable=true" + - "traefik.http.routers.cyberstrawberry-onlyoffice.rule=Host(`office.cyberstrawberry.net`)" + - "traefik.http.routers.cyberstrawberry-onlyoffice.entrypoints=websecure" + - "traefik.http.routers.cyberstrawberry-onlyoffice.tls.certresolver=myresolver" + - "traefik.http.services.cyberstrawberry-onlyoffice.loadbalancer.server.port=80" + - "traefik.http.routers.cyberstrawberry-onlyoffice.middlewares=onlyoffice-headers" + - "traefik.http.middlewares.onlyoffice-headers.headers.customrequestheaders.X-Forwarded-Proto=https" + #- "traefik.http.middlewares.onlyoffice-headers.headers.accessControlAllowOrigin=*" + environment: + - JWT_ENABLED=true + - JWT_SECRET=REDACTED #SET THIS TO SOMETHING SECURE + restart: always + networks: + docker_network: + ipv4_address: 192.168.5.143 +networks: + default: + external: + name: docker_network + docker_network: + external: true +``` + +```yaml title=".env" +Not Applicable +``` + +!!! tip "Tip" + If you wish to use this in a non-commercial homelab environment without limits, [this script](https://wiki.muwahhid.ru/ru/Unraid/Docker/Onlyoffice-Document-Server) does an endless trial without functionality limits. + + ```text + docker stop office-document-server-ee + docker rm office-document-server-ee + rm -r /mnt/user/appdata/onlyoffice/DocumentServer + sleep 5 + + ``` + + Docker Run Method: + + ```text + docker run -d --name='office-document-server-ee' --net='bridge' -e TZ="Europe/Moscow" -e HOST_OS="Unraid" -e 'JWT_ENABLED'='true' -e 'JWT_SECRET'='mySecret' -p '8082:80/tcp' -p '4432:443/tcp' -v '/mnt/user/appdata/onlyoffice/DocumentServer/logs':'/var/log/onlyoffice':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/data':'/var/www/onlyoffice/Data':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/lib':'/var/lib/onlyoffice':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/db':'/var/lib/postgresql':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/fonts':'/usr/share/fonts/truetype/custom':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/forgotten':'/var/lib/onlyoffice/documentserver/App_Data/cache/files/forgotten':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/rabbitmq':'/var/lib/rabbitmq':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/redis':'/var/lib/redis':'rw' 'onlyoffice/documentserver-ee' + ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Files and Collaboration Documentation](<../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/Media and Gaming/Pyload.md b/deployments/Applications/Files and Collaboration/Pyload.md similarity index 78% rename from deployments/services/Media and Gaming/Pyload.md rename to deployments/Applications/Files and Collaboration/Pyload.md index 42f776d..87681cb 100644 --- a/deployments/services/Media and Gaming/Pyload.md +++ b/deployments/Applications/Files and Collaboration/Pyload.md @@ -6,11 +6,11 @@ tags: - Docker --- -**Purpose**: pyLoad-ng is a Free and Open Source download manager written in Python and designed to be extremely lightweight, easily extensible and fully manageable via web. +## Purpose +pyLoad-ng is a Free and Open Source download manager written in Python and designed to be extremely lightweight, easily extensible and fully manageable via web. [Detailed LinuxServer.io Deployment Info](https://docs.linuxserver.io/images/docker-pyload-ng/) - ## Docker Configuration ```yaml title="docker-compose.yml" version: '3.9' @@ -46,7 +46,7 @@ networks: docker_network: external: true ``` - + 1. Set this to your own timezone. 2. This is optional. Additional documentation needed to convey what this port is used for. Possibly API access. 3. This assumes you want your download folder to be a SMB network share, this section allows you to connect to the share so Pyload can download content directly into the network folder. Replace the username and `REDACTED` password with your actual credentials. Remove the `domain` argument if the SMB server is not domain-joined. @@ -61,6 +61,7 @@ N/A ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -83,4 +84,9 @@ http: !!! warning "Change Default Admin Credentials" Pyload ships with the username `pyload` and password `pyload`. Make sure you change the credentials immediately after initial login. - Navigate to "**Settings > Users > Pyload:"Change Password"**" \ No newline at end of file + Navigate to "**Settings > Users > Pyload:"Change Password"**" + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Files and Collaboration Documentation](<../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/productivity/Stirling PDF.md b/deployments/Applications/Files and Collaboration/Stirling PDF.md similarity index 59% rename from deployments/services/productivity/Stirling PDF.md rename to deployments/Applications/Files and Collaboration/Stirling PDF.md index cc358ce..b394fb4 100644 --- a/deployments/services/productivity/Stirling PDF.md +++ b/deployments/Applications/Files and Collaboration/Stirling PDF.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: This is a powerful locally hosted web based PDF manipulation tool using docker that allows you to perform various operations on PDF files, such as splitting merging, converting, reorganizing, adding images, rotating, compressing, and more. This locally hosted web application started as a 100% ChatGPT-made application and has evolved to include a wide range of features to handle all your PDF needs. +## Purpose +This is a powerful locally hosted web based PDF manipulation tool using docker that allows you to perform various operations on PDF files, such as splitting merging, converting, reorganizing, adding images, rotating, compressing, and more. This locally hosted web application started as a 100% ChatGPT-made application and has evolved to include a wide range of features to handle all your PDF needs. ## Docker Configuration ```yaml title="docker-compose.yml" @@ -45,6 +46,7 @@ N/A ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -63,4 +65,9 @@ http: servers: - url: http://192.168.5.54:8080 passHostHeader: true -``` \ No newline at end of file +``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Files and Collaboration Documentation](<../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/productivity/trilium.md b/deployments/Applications/Files and Collaboration/Trilium.md similarity index 55% rename from deployments/services/productivity/trilium.md rename to deployments/Applications/Files and Collaboration/Trilium.md index f1252ec..e37c19c 100644 --- a/deployments/services/productivity/trilium.md +++ b/deployments/Applications/Files and Collaboration/Trilium.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Build your personal knowledge base with [Trilium Notes](https://github.com/zadam/trilium/tree/master). +## Purpose +Build your personal knowledge base with [Trilium Notes](https://github.com/zadam/trilium/tree/master). ```yaml title="docker-compose.yml" version: '2.1' @@ -34,7 +35,7 @@ networks: N/A ``` -# Traefik Configuration +## Traefik Configuration ```yaml title="notes.bunny-lab.io.yml" http: routers: @@ -53,4 +54,9 @@ http: servers: - url: http://192.168.5.11:8080 passHostHeader: true -``` \ No newline at end of file +``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Files and Collaboration Documentation](<../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/Applications/Files and Collaboration/Windows Server/DFS Namespaces with Replication.md b/deployments/Applications/Files and Collaboration/Windows Server/DFS Namespaces with Replication.md new file mode 100644 index 0000000..783ac98 --- /dev/null +++ b/deployments/Applications/Files and Collaboration/Windows Server/DFS Namespaces with Replication.md @@ -0,0 +1,134 @@ +--- +tags: + - DFS + - Windows Server + - Windows + - File Services +--- + +## Purpose +If you want data available from a single, consistent UNC path while hosting it on multiple file servers, use **DFS Namespaces (DFSN)**. A namespace presents a *virtual* folder tree (for example, `\\bunny-lab.io\Projects`) whose folders point to one or more **folder targets** (actual SMB shares on your servers). +**DFS Replication (DFSR)** is a *separate* feature you configure to keep the contents of those targets in sync. + +This document walks through creating a domain-based DFS namespace and enabling DFS Replication for two servers. + +!!! info "Assumptions" + You have two Windows Server machines (e.g., `LAB-FPS-01` and `LAB-FPS-02`) running an edition that supports DFS (Standard or Datacenter), both activated, domain-joined, and using static IPs. + +### Installing Server Roles +Install the roles on **both servers**: + +- **Server Manager → Manage → Add Roles and Features** +- Click **Next** to **Server Roles** +- Expand **File and Storage Services** + - Expand **File and iSCSI Services** + - Check **File Server** + - Check **DFS Namespaces** + - Check **DFS Replication** +- **Next → Next → Install**, then finish. + +### Create and Configure Network Shares +Create (or identify) the folders you want to publish in the namespace, and share them on **each** server. Be sure to enable **Access-based Enumeration** on all of the folder shares for additional security. You only need to ensure that the files exist on one of the file servers,then you need to create empty top-level folders with the same names on the replica servers, data will be replicated automatically from the file server to the empty folders. + +Additionally, it is recommended (if possible) to set the share names to be hidden. For example `\\LAB-FPS-01\Projects$`, that way it ensures that users access the share via DFS at `\\bunny-lab.io\Projects` and users don't accidentally access the network shares directly, bypassing DFS. For example, the local path would be `Z:\Projects` but the network share would be `\\LAB-FPS-01\Projects$`. *This wouldn't break things like replication, but it would muck things up a little bit organizationally. The data would still be replicated between both servers, we just dont want users using direct server shares like that, which bypasses the high-availability and load-balancing features of DFS* + +!!! warning "What must match vs. what can differ" + - **Must exist on each server:** a shared folder to act as the *folder target* (path can differ per server). + - **Share permissions:** are **not replicated**; set them on each server. + - **NTFS permissions inside the replicated folder:** **are replicated** by DFSR and should be consistent. + - Targets do **not** have to use identical share names/paths, but keeping them consistent simplifies things. + +| **Permission Type** | **User / Group** | **Access** | Level** | +| :---- | :---- | :---- | :---- | +| Share | `Everyone` (or `Authenticated Users`) | Full Control | Best practice is to grant broad Full Control on the **share** and enforce access with NTFS. | +| NTFS | `SYSTEM` | Full Control | Required for DFSR service. | +| NTFS | `Share_Admins` | Full Control | Optional admin group for data management. | +| NTFS | *Business groups needing access* | Modify | Grant least privilege to required users/groups. | + +!!! info "Note On Inheritance" + Disabling inheritance is **not required** for DFS/DFSR. Keep it enabled unless you have a clear reason to flatten ACLs; inheritance often reduces long-term admin overhead. + +### DFS Breakdown +A **namespace** is a logical view like `\\bunny-lab.io\Projects`. Inside it, you create DFS **folders** (e.g., `Scripting`) that point to one or more **folder targets**, such as: + +- `\\LAB-FPS-01\Projects$\Scripting` +- `\\LAB-FPS-02\Projects$\Scripting` + +The namespace root itself isn't where you store data; it's a directory of links. Place data in the folder targets the DFS folder points to. + +### DFS Configuration +You can run these steps from either server (or any admin workstation with the RSAT tools). DFSN configuration is stored in AD and on namespace servers and applies across members automatically. + +#### Create Namespace +- **Server Manager → Tools → DFS Management** +- Right-click **Namespaces** → **New Namespace...** + - Choose a server to host the namespace (e.g., `LAB-FPS-01`) → **Next** + - Name the namespace (e.g., `Projects`) → **Next** + - You can leave **Edit Settings** at defaults; those control the local folder that backs the namespace root, not your data. + - Choose **Domain-based namespace** and check **Enable Windows Server 2008 mode** (required for larger scale and Access-based enumeration). + - Resulting path: `\\bunny-lab.io\Projects` + - **Next → Create** + +#### Make Namespace Highly-Available +We have to perform an extra step to ensure that every file server can act as within a multi-master context, allowing for high availability. To do this in this example, we will add `LAB-FPS-02` as a secondary namespace server for every namespace that we create. + +- Right-Click **DFS Management** > **Namespaces** > `\\bunny-lab.io\Projects` +- Click **Add Namespace Server...** +- Under "Namespace Server" enter `LAB-FPS-02` then click **OK**. + +#### Enable Access-Based Enumeration on Namespace +- Right-Click **DFS Management** > **Namespaces** > `\\bunny-lab.io\Projects` +- Click **Properties** +- Click **Advanced** +- Check **Enable access-based enumeration for this namespace** +- Click **OK** + +#### Link Folders to Namespace +Create the DFS folders and add folder targets: + +- Right-click the new namespace (e.g., `\\bunny-lab.io\Projects`) → **New Folder...** + - **Name:** `Scripting` + - **Add** folder targets (one per server), e.g.: + - `\\LAB-FPS-01\Projects$\Scripting` + - `\\LAB-FPS-02\Projects$\Scripting` + - You can simply copy-paste the previous server location and substitute the hostname (e.g. switching `01` to `02`) instead of browsing for the folder. + - You *may* be prompted to create the folder because it does not exist on `LAB-FPS-02`, in this circumstance, you can tell it to create the folder automatically with read-only permissions. *Don't worry, when replication from `LAB-FPS-01` occurs, NTFS permissions will be overwritten to the correct users and groups.* + - When prompted *"Create a replication group to synchronize the folder targets?"*, click **Yes** to launch the DFS Replication wizard. + +!!! info "**Be patient**" + The Replication wizard can take ~1 minute to appear. + +#### Configure Replication Group +In the Replication wizard that appears after about a minute, you can configure the replication group for the folder: + +!!! bug "If Wizard did Not Appear (or Crashed)" + In my homelab testing, I had two times when the wizard crashed or simply never opened. If this happens to you, you can manually re-trigger the wizard for the target folder by right-clicking the folder (e.g. `\\bunny-lab.io\Projects\Scripting`) and selecting **Replicate Folder**. + +- **Replication Group Name**: *(leave as suggested)* +- **Replicated Folder Name**: *(leave as suggested)* +- **Next → Next** +- **Primary member**: pick the server with the **most up-to-date** copy of the data (e.g., `LAB-FPS-01`). + +!!! abstract "Replication Behavior and Expectations" + When you first create a replication group, DFSR needs a baseline copy of the data to start from. You designate one server as the Primary Member to serve as that baseline. (e.g. `LAB-FPS-01`) During the first sync, DFSR assumes that whatever exists on the primary member's folder is the "truth." So if the same file exists on another server (e.g. `LAB-FPS-02`) but with different timestamps, sizes, or hashes, the primary member's copy wins - but only during this first synchronization. After that initial sync is complete, the "primary" flag loses all authority. Replication becomes multi-master, meaning every member can make changes, and DFSR uses its conflict resolution algorithm (based on version vectors, update sequence numbers, and timestamps) to decide which change wins going forward. In other words, no server remains “the boss” after initialization. Files unique to other member servers that only exist on them will not be wiped and will be replicated across all member servers including the primary member. + +- **Topology**: `Full mesh` (good for two servers; for many sites, consider hub-and-spoke). +- **Replication schedule**: leave **Full** (24x7) unless you need bandwidth windows. +- **Create** + +!!! success "Replication group created" + You should see green ticks for the following. Give everything some time to replicate as it depends on active directory replication speeds to push out the configuration across the DFS member servers and begin the replication. + + - ✅Create replication group + - ✅Create members + - ✅Update folder security + - ✅Create replicated folder + - ✅Create membership objects + - ✅Update folder properties + - ✅Create connections + +## Validate and Maintain DFS +Use [the namespace and replication report](<../../../../scripts/Applications/Files and Collaboration/DFS/Report DFS Namespaces and Replication.md>) to inspect the deployed structure, then [check the directional replication backlog](<../../../../scripts/Applications/Files and Collaboration/DFS/Report DFS Replication Backlog.md>). If the management console shows inconsistent objects, follow [the console repair workflow](<../../../../workflows/Applications/Files and Collaboration/DFS/Repair an Inconsistent DFS Management Console.md>). + +## Related Documentation +- [Related Files and Collaboration Documentation](<../../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/productivity/wordpress.md b/deployments/Applications/Files and Collaboration/Wordpress.md similarity index 65% rename from deployments/services/productivity/wordpress.md rename to deployments/Applications/Files and Collaboration/Wordpress.md index a484425..f29accd 100644 --- a/deployments/services/productivity/wordpress.md +++ b/deployments/Applications/Files and Collaboration/Wordpress.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: At its core, WordPress is the simplest, most popular way to create your own website or blog. In fact, WordPress powers over 43.3% of all the websites on the Internet. Yes – more than one in four websites that you visit are likely powered by WordPress. +## Purpose +At its core, WordPress is the simplest, most popular way to create your own website or blog. In fact, WordPress powers over 43.3% of all the websites on the Internet. Yes – more than one in four websites that you visit are likely powered by WordPress. ```yaml title="docker-compose.yml" version: '3.7' @@ -54,3 +55,7 @@ networks: WORDPRESS_DB_PASSWORD=SecurePassword101 MYSQL_ROOT_PASSWORD=SecurePassword202 ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Files and Collaboration Documentation](<../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/gaming/Ark Survival Ascended.md b/deployments/Applications/Gaming and Media/Ark Survival Ascended.md similarity index 90% rename from deployments/services/gaming/Ark Survival Ascended.md rename to deployments/Applications/Gaming and Media/Ark Survival Ascended.md index 30a1ed3..8efd038 100644 --- a/deployments/services/gaming/Ark Survival Ascended.md +++ b/deployments/Applications/Gaming and Media/Ark Survival Ascended.md @@ -4,7 +4,7 @@ tags: - Gaming --- -**Purpose**: +## Purpose This document outlines some of the prerequisites as well as deployment process for an ARK: Survival Ascended Server ## Prerequisites @@ -13,20 +13,20 @@ We need to install the Visual C++ Redistributable for both x86 and x64 - [Download Visual C++ Redistributable (x64)](https://aka.ms/vs/17/release/vc_redist.x64.exe) - [Download Visual C++ Redistributable (x86)](https://aka.ms/vs/17/release/vc_redist.x86.exe) - ## Run Unreal Engine Certificate Trust Script There is an issue where if you run a dedicated server, part of that requires API access to Epic Games and that will not work without installing a few certificates. The original Github page can be found [here](https://github.com/Ch4r0ne/UnrealEngine_Dedicated_Server_Install_CA/tree/main), which details the reason for it in more detail. !!! note "Run as Administrator" You need to run the command as an administrator. This command will download the script automatically and temporarily bypass the script execution policy to run the script: - ``` + + ```text PowerShell -ExecutionPolicy Bypass -Command "irm 'https://raw.githubusercontent.com/Ch4r0ne/UnrealEngine_Dedicated_Server_Install_CA/main/Install_Certificate.ps1' | iex" ``` ## SteamCMD Deployment Script You will need to make a folder somewhere on the computer, such as the desktop, and name it something like "ARK Updater", then put the following script into it. You will need to run this script before you can proceed to the next step. -```jsx title="C:\Users\nicole.rappe\Desktop\ARK_Updater\Update_Server.bat" +```text title="C:\Users\nicole.rappe\Desktop\ARK_Updater\Update_Server.bat" @echo off set STEAMCMDDIR="C:\SteamCMD\" set SERVERDIR="C:\ASAServer\" @@ -43,19 +43,19 @@ exit ## Launch Script Now you need to configure a launch script to actually start the dedicated server. This can be placed anywhere, but I suggest putting it into `C:\asaserver\ShooterGame\Saved` along with the world save data. -```jsx title="C:\asaserver\ShooterGame\Saved\Launch_Server.bat" +```text title="C:\asaserver\ShooterGame\Saved\Launch_Server.bat" @echo off start C:\asaserver\ShooterGame\Binaries\Win64\ArkAscendedServer.exe ScorchedEarth_WP?listen?SessionName=BunnyLab?Port=7777?QueryPort=27015?ServerPassword=SomethingSecure?ServerAdminPassword=SomethingVerySecure -WinLiveMaxPlayers=50 -log -crossplay-enable-pc -crossplay-enable-wingdk -mods=928548,928621,928597,928818,929543,937546,930684,930404,940022,941697,930851,948051,932365,929420,967786,930494 exit ``` !!! tip "Adding Mods" - When you are adding mods, you will notice they are found on [CurseForge](https://www.curseforge.com/ark-survival-ascended). When you are looking for the mod ID, it is actually listed under CurseForge as the `Project ID`. Just copy that number and put it in a comma-separated list such as what is seen in the example above. + When you are adding mods, you will notice they are found on [CurseForge](https://www.curseforge.com/ark-survival-ascended). When you are looking for the mod ID, it is actually listed under CurseForge as the `Project ID`. Just copy that number and put it in a comma-separated list such as what is seen in the example above. ## Dump Configuration .ini Files At this point, you will want to launch the server and have someone join it so it can generate the necessary world files / configuration data. Then you will run the following commands in the console (from the server hosting the ARK server) in order to dump the configuration (ini) files to disk. -``` +```text enablecheats cheat SaveWorld cheat DoExit @@ -69,4 +69,7 @@ You will find the dumped configuration files at `C:\asaserver\ShooterGame\Saved\ !!! info "Optional: Generate Files from Singleplayer World" You may want to start a singleplayer world and set all of the configuration variables to your desired values, then load into the world. Once you have made landfall, quit out of the game to shut down the singleplayer world. - From this point, you can find your `Game.ini` and `GameUserSettings.ini` files in `steamapps\common\ARK Survival Ascended\ShooterGame\Saved\Config\Windows`. Simply copy these two files into your server's configuration folder located at `C:\asaserver\ShooterGame\Saved\Config\WindowsServer` and launch the server. \ No newline at end of file + From this point, you can find your `Game.ini` and `GameUserSettings.ini` files in `steamapps\common\ARK Survival Ascended\ShooterGame\Saved\Config\Windows`. Simply copy these two files into your server's configuration folder located at `C:\asaserver\ShooterGame\Saved\Config\WindowsServer` and launch the server. + +## Related Documentation +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/Media and Gaming/Emulatorjs.md b/deployments/Applications/Gaming and Media/Emulatorjs.md similarity index 62% rename from deployments/services/Media and Gaming/Emulatorjs.md rename to deployments/Applications/Gaming and Media/Emulatorjs.md index 25e8e97..1ec27df 100644 --- a/deployments/services/Media and Gaming/Emulatorjs.md +++ b/deployments/Applications/Gaming and Media/Emulatorjs.md @@ -6,7 +6,8 @@ tags: - Docker --- -**Purpose**: Emulatorjs is a browser web based emulation portable to nearly any device for many retro consoles. A mix of emulators is used between Libretro and EmulatorJS. +## Purpose +Emulatorjs is a browser web based emulation portable to nearly any device for many retro consoles. A mix of emulators is used between Libretro and EmulatorJS. ## Docker Configuration ```yaml title="docker-compose.yml" @@ -43,6 +44,7 @@ N/A ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -65,4 +67,9 @@ http: !!! note Port 80 = Frontend - Port 3000 = Management Backend \ No newline at end of file + Port 3000 = Management Backend + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/gaming/pterodactyl.md b/deployments/Applications/Gaming and Media/Pterodactyl.md similarity index 62% rename from deployments/services/gaming/pterodactyl.md rename to deployments/Applications/Gaming and Media/Pterodactyl.md index 58c36fe..180c024 100644 --- a/deployments/services/gaming/pterodactyl.md +++ b/deployments/Applications/Gaming and Media/Pterodactyl.md @@ -4,18 +4,21 @@ tags: - Gaming --- -**Purpose**: Pterodactyl is the open-source game server management panel built with PHP, React, and Go. Designed with security in mind, Pterodactyl runs all game servers in isolated Docker containers while exposing a beautiful and intuitive UI to administrators and users. +## Purpose +Pterodactyl is the open-source game server management panel built with PHP, React, and Go. Designed with security in mind, Pterodactyl runs all game servers in isolated Docker containers while exposing a beautiful and intuitive UI to administrators and users. [Official Website](https://pterodactyl.io/panel/1.0/getting_started.html) !!! note This documentation assumes you are running Rocky Linux 9.3 or higher. **Install EPEL Repository and other tools**: + ```bash sudo yum -y install epel-release curl ca-certificates gnupg ``` **Add Redis Repository**: + ```bash sudo rpm --import https://packages.redis.io/gpg echo "[redis6] @@ -27,26 +30,33 @@ gpgkey=https://packages.redis.io/gpg" | sudo tee /etc/yum.repos.d/redis.repo ``` **Add MariaDB Repository**: + ```bash sudo curl -LsS https://downloads.mariadb.com/MariaDB/mariadb_repo_setup | sudo bash ``` **Update Repositories List**: + ```bash sudo yum update ``` **Install Dependencies**: -Before installing PHP, check the available PHP versions in your enabled repositories. Install PHP and other dependencies as follows: +Before installing PHP, check the available PHP versions in your enabled repositories. Install PHP and other dependencies as follows: + ```bash sudo yum -y install php php-{common,cli,gd,mysql,mbstring,bcmath,xml,fpm,curl,zip} mariadb-server nginx tar unzip git redis ``` 7. **Installing Composer**: + ```bash curl -sS https://getcomposer.org/installer | php sudo mv composer.phar /usr/local/bin/composer chmod +x /usr/local/bin/composer ``` -This script should work well with Rocky Linux and similar RHEL-based distributions, using `yum` for package management. However, keep in mind that package names and versions may vary between repositories, so you might need to adjust them based on what's available in your system's repositories. \ No newline at end of file +This script should work well with Rocky Linux and similar RHEL-based distributions, using `yum` for package management. However, keep in mind that package names and versions may vary between repositories, so you might need to adjust them based on what's available in your system's repositories. + +## Related Documentation +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/gaming/valheim.md b/deployments/Applications/Gaming and Media/Valheim.md similarity index 82% rename from deployments/services/gaming/valheim.md rename to deployments/Applications/Gaming and Media/Valheim.md index 9c1ecb2..bf326a8 100644 --- a/deployments/services/gaming/valheim.md +++ b/deployments/Applications/Gaming and Media/Valheim.md @@ -4,7 +4,7 @@ tags: - Gaming --- -**Purpose**: +## Purpose This document outlines some of the prerequisites as well as deployment process for an dedicated Valheim server. ## Prerequisites @@ -16,7 +16,7 @@ We need to install the Visual C++ Redistributable for both x86 and x64 ## SteamCMD Deployment Script You will need to make a folder somewhere on the computer, such as the desktop, and name it something like "ARK Updater", then put the following script into it. You will need to run this script before you can proceed to the next step. -```jsx title="C:\Users\nicole.rappe\Downloads\SteamCMD\Update_Server.bat" +```text title="C:\Users\nicole.rappe\Downloads\SteamCMD\Update_Server.bat" @echo off steamcmd.exe +force_install_dir "C:\Valheim_Dedicated_Server" +login anonymous +app_update 896660 -beta public validate +quit ``` @@ -24,7 +24,7 @@ steamcmd.exe +force_install_dir "C:\Valheim_Dedicated_Server" +login anonymous + ## Launch Script Now you need to configure a launch script to actually start the dedicated server. This can be placed anywhere, but I suggest putting it into `C:\asaserver\ShooterGame\Saved` along with the world save data. -```jsx title="C:\valheim_dedicated_server\Launch_Server.bat" +```text title="C:\valheim_dedicated_server\Launch_Server.bat" @echo off set SteamAppId=892970 @@ -36,4 +36,7 @@ valheim_server -nographics -batchmode -name "Bunny Lab" -port 2456 -world "Dedic !!! warning "Launch Script Considerations" - Make a local copy of this script to avoid it being overwritten by steam. - Minimum password length is 5 characters & Password cant be in the server name. - - You need to make sure the ports TCP/UDP 2456-2457 is being forwarded to your server through your server VM & firewall. \ No newline at end of file + - You need to make sure the ports TCP/UDP 2456-2457 is being forwarded to your server through your server VM & firewall. + +## Related Documentation +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/Home and IOT/Frigate.md b/deployments/Applications/Home Automation/Frigate.md similarity index 71% rename from deployments/services/Home and IOT/Frigate.md rename to deployments/Applications/Home Automation/Frigate.md index 7318091..1936898 100644 --- a/deployments/services/Home and IOT/Frigate.md +++ b/deployments/Applications/Home Automation/Frigate.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: A complete and local NVR designed for Home Assistant with AI object detection. Uses OpenCV and Tensorflow to perform realtime object detection locally for IP cameras. +## Purpose +A complete and local NVR designed for Home Assistant with AI object detection. Uses OpenCV and Tensorflow to perform realtime object detection locally for IP cameras. ```yaml title="docker-compose.yml" version: "3.9" @@ -35,7 +36,7 @@ services: FRIGATE_RTSP_PASSWORD: ${FRIGATE_RTSP_PASSWORD} networks: docker_network: - ipv4_address: 192.168.5.201 + ipv4_address: 192.168.5.201 mqtt: container_name: mqtt @@ -45,7 +46,7 @@ services: networks: docker_network: ipv4_address: 192.168.5.202 - + networks: docker_network: external: true @@ -54,3 +55,7 @@ networks: ```yaml title=".env" FRIGATE_RTSP_PASSWORD=SomethingSecure101 ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/Home and IOT/HomeAssistant.md b/deployments/Applications/Home Automation/HomeAssistant.md similarity index 51% rename from deployments/services/Home and IOT/HomeAssistant.md rename to deployments/Applications/Home Automation/HomeAssistant.md index fa9adab..9af55ac 100644 --- a/deployments/services/Home and IOT/HomeAssistant.md +++ b/deployments/Applications/Home Automation/HomeAssistant.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Open source home automation that puts local control and privacy first. Powered by a worldwide community of tinkerers and DIY enthusiasts. +## Purpose +Open source home automation that puts local control and privacy first. Powered by a worldwide community of tinkerers and DIY enthusiasts. ```yaml title="docker-compose.yml" version: '3' @@ -42,3 +43,10 @@ networks: ```yaml title=".env" Not Applicable ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Docker Macvlan Subinterface](<../../../workflows/Containers/Docker/Create a Macvlan Subinterface.md>) — Review the network setup documented for the home-automation environment. +- [Tuya Device Integration](<../../../workflows/Applications/Home Automation/Connect Tuya Smart Lights.md>) — Find the device reservations and local integration notes. +- [Frigate]() — Find the separately documented camera service. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/monitoring/gatus.md b/deployments/Applications/Monitoring/Gatus.md similarity index 71% rename from deployments/services/monitoring/gatus.md rename to deployments/Applications/Monitoring/Gatus.md index 9a6e672..d911b5e 100644 --- a/deployments/services/monitoring/gatus.md +++ b/deployments/Applications/Monitoring/Gatus.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Gatus Service Status Server. +## Purpose +Gatus Service Status Server. ## Docker Configuration ```yaml title="docker-compose.yml" @@ -41,9 +42,9 @@ services: depends_on: postgres: condition: service_healthy - dns: - - 192.168.3.25 - - 192.168.3.26 + dns: + - 192.168.3.25 + - 192.168.3.26 networks: docker_network: ipv4_address: 192.168.5.8 @@ -59,6 +60,7 @@ N/A ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -78,4 +80,9 @@ http: servers: - url: http://192.168.5.8:8080 passHostHeader: true -``` \ No newline at end of file +``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/notifications/ntfy.md b/deployments/Applications/Monitoring/Ntfy.md similarity index 56% rename from deployments/services/notifications/ntfy.md rename to deployments/Applications/Monitoring/Ntfy.md index 0f8d55f..bf7a747 100644 --- a/deployments/services/notifications/ntfy.md +++ b/deployments/Applications/Monitoring/Ntfy.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: ntfy (pronounced notify) is a simple HTTP-based pub-sub notification service. It allows you to send notifications to your phone or desktop via scripts from any computer, and/or using a REST API. It's infinitely flexible, and 100% free software. +## Purpose +ntfy (pronounced notify) is a simple HTTP-based pub-sub notification service. It allows you to send notifications to your phone or desktop via scripts from any computer, and/or using a REST API. It's infinitely flexible, and 100% free software. ```yaml title="docker-compose.yml" version: "2.1" @@ -41,3 +42,7 @@ networks: ```yaml title=".env" Not Applicable ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/monitoring/Speedtest Tracker.md b/deployments/Applications/Monitoring/Speedtest Tracker.md similarity index 83% rename from deployments/services/monitoring/Speedtest Tracker.md rename to deployments/Applications/Monitoring/Speedtest Tracker.md index f7e2b39..641a47a 100644 --- a/deployments/services/monitoring/Speedtest Tracker.md +++ b/deployments/Applications/Monitoring/Speedtest Tracker.md @@ -5,8 +5,8 @@ tags: - Docker --- -## Purpose: -Speedtest Tracker is a self-hosted application that monitors the performance and uptime of your internet connection over time. +## Purpose +Speedtest Tracker is a self-hosted application that monitors the performance and uptime of your internet connection over time. [Detailed Configuration Reference](https://docs.speedtest-tracker.dev/getting-started/installation) ## Docker Configuration @@ -43,7 +43,7 @@ services: networks: docker_network: ipv4_address: 192.168.5.38 - + db: image: postgres:17 restart: always @@ -61,7 +61,7 @@ services: timeout: 5s networks: docker_network: - ipv4_address: 192.168.5.39 + ipv4_address: 192.168.5.39 networks: default: external: @@ -74,7 +74,7 @@ networks: 2. You can generate a secure appkey with the following command: `echo -n 'base64:'; openssl rand -base64 32;` > Copy this key including the `base64:` prefix and paste it as your APP_KEY environment variable value. -3. This restricts the speedtest target to a specific speedtest server. In this example, it is a Missoula, MT speedtest server. You can get these codes from the yellow Speedtest button menu in the WebUI and then come back and redeploy the stack with the number entered here. +3. This restricts the speedtest target to a specific speedtest server. In this example, it is a Missoula, MT speedtest server. You can get these codes from the yellow Speedtest button menu in the WebUI and then come back and redeploy the stack with the number entered here. ```yaml title=".env" DB_PASSWORD=SecurePassword @@ -87,6 +87,7 @@ BASE64_APPKEY=SECUREAPPKEY ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -105,4 +106,9 @@ http: servers: - url: http://192.168.5.38:80 passHostHeader: true -``` \ No newline at end of file +``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/monitoring/uptimekuma.md b/deployments/Applications/Monitoring/UptimeKuma.md similarity index 64% rename from deployments/services/monitoring/uptimekuma.md rename to deployments/Applications/Monitoring/UptimeKuma.md index f63b881..6b5f452 100644 --- a/deployments/services/monitoring/uptimekuma.md +++ b/deployments/Applications/Monitoring/UptimeKuma.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Deploy Uptime Kuma uptime monitor to monitor services in the homelab and send notifications to various services. +## Purpose +Deploy Uptime Kuma uptime monitor to monitor services in the homelab and send notifications to various services. ```yaml title="docker-compose.yml" version: '3' @@ -38,3 +39,7 @@ networks: ```yaml title=".env" Not Applicable ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/Security and Utility/Changedetection.md b/deployments/Applications/Utilities/Changedetection.md similarity index 65% rename from deployments/services/Security and Utility/Changedetection.md rename to deployments/Applications/Utilities/Changedetection.md index 4892904..1451c98 100644 --- a/deployments/services/Security and Utility/Changedetection.md +++ b/deployments/Applications/Utilities/Changedetection.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Detect website content changes and perform meaningful actions - trigger notifications via Discord, Email, Slack, Telegram, API calls and many more. +## Purpose +Detect website content changes and perform meaningful actions - trigger notifications via Discord, Email, Slack, Telegram, API calls and many more. ## Docker Configuration ```yaml title="docker-compose.yml" @@ -13,7 +14,7 @@ version: "3.8" services: app: image: dgtlmoon/changedetection.io - container_name: changedetection.io + container_name: changedetection.io environment: - TZ=America/Denver volumes: @@ -39,13 +40,14 @@ networks: external: true ``` -```jsx title=".env" +```ini title=".env" N/A ``` ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. -``` yaml + +```yaml http: routers: changedetection: @@ -63,4 +65,9 @@ http: servers: - url: http://192.168.5.49:5000 passHostHeader: true -``` \ No newline at end of file +``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/Applications/Utilities/Cyberchef.md b/deployments/Applications/Utilities/Cyberchef.md new file mode 100644 index 0000000..0d9fbb4 --- /dev/null +++ b/deployments/Applications/Utilities/Cyberchef.md @@ -0,0 +1,40 @@ +--- +tags: + - CyberChef + - Security + - Docker +--- + +## Purpose +The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis. + +```yaml title="docker-compose.yml" +version: "3.8" +services: + app: + image: mpepping/cyberchef:latest + container_name: cyberchef + environment: + - TZ=America/Denver + ports: + - 8000:8000 + restart: always + networks: + docker_network: + ipv4_address: 192.168.5.55 + +networks: + default: + external: + name: docker_network + docker_network: + external: true +``` + +```ini title=".env" +N/A +``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/Applications/Utilities/IT Tools.md b/deployments/Applications/Utilities/IT Tools.md new file mode 100644 index 0000000..6dd2eda --- /dev/null +++ b/deployments/Applications/Utilities/IT Tools.md @@ -0,0 +1,38 @@ +--- +tags: + - IT-Tools + - Security + - Docker +--- + +## Purpose +Collection of handy online tools for developers, with great UX. + +```yaml title="docker-compose.yml" +version: "3" + +services: + server: + image: corentinth/it-tools:latest + container_name: it-tools + environment: + - TZ=America/Denver + restart: always + ports: + - "80:80" + networks: + docker_network: + ipv4_address: 192.168.5.16 + +networks: + docker_network: + external: true +``` + +```yaml title=".env" +Not Applicable +``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/Security and Utility/Searx.md b/deployments/Applications/Utilities/Searx.md similarity index 67% rename from deployments/services/Security and Utility/Searx.md rename to deployments/Applications/Utilities/Searx.md index 3ac443f..fd9fd3a 100644 --- a/deployments/services/Security and Utility/Searx.md +++ b/deployments/Applications/Utilities/Searx.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Deploys a SearX Meta Search Engine Server +## Purpose +Deploys a SearX Meta Search Engine Server ## Docker Configuration ```yaml title="docker-compose.yml" @@ -38,6 +39,7 @@ Not Applicable ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -55,4 +57,9 @@ http: servers: - url: http://192.168.5.124:8080 passHostHeader: true -``` \ No newline at end of file +``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/Applications/index.md b/deployments/Applications/index.md new file mode 100644 index 0000000..1e47f08 --- /dev/null +++ b/deployments/Applications/index.md @@ -0,0 +1,24 @@ +--- +tags: + - Applications + - Deployments + - Documentation +--- + +# Applications +## Purpose +Find deployments for applications. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- Asset Management +- Communication +- Dashboards +- Email +- Files and Collaboration +- Gaming and Media +- Home Automation +- Monitoring +- Utilities + +## Follow the Subject +[Applications](<../../reference/Applications/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/deployments/services/backup/kopia.md b/deployments/Backup and Recovery/Kopia.md similarity index 74% rename from deployments/services/backup/kopia.md rename to deployments/Backup and Recovery/Kopia.md index 14b81a1..7dfcbbd 100644 --- a/deployments/services/backup/kopia.md +++ b/deployments/Backup and Recovery/Kopia.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included. +## Purpose +Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included. ```yaml title="docker-compose.yml" version: '3.7' @@ -39,12 +40,16 @@ networks: docker_network: external: true ``` + !!! note "Credentials" Your username will be `kopia@kopia-backup` and the password will be the value you set for `--server-password` in the entrypoint section of the compose file. The `KOPIA_PASSWORD:` is used by the backup repository, such as Backblaze B2, to encrypt/decrypt the backed-up data, and must be updated in the compose file if the repository is changed / updated. - ```yaml title=".env" KOPIA_ENRYPTION_PASSWORD=PasswordUsedToEncryptDataOnBackblazeB2 KOPIA_SERVER_PASSWORD=ThisIsUsedToLogIntoKopiaWebUI KOPIA_SERVER_USERNAME=kopia@kopia-backup ``` + +## Related Documentation +- [Docker Network Prerequisite](<../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Backup and Recovery Documentation](<../../reference/Backup and Recovery/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/Backup and Recovery/index.md b/deployments/Backup and Recovery/index.md new file mode 100644 index 0000000..bcad452 --- /dev/null +++ b/deployments/Backup and Recovery/index.md @@ -0,0 +1,16 @@ +--- +tags: + - Backup and Recovery + - Deployments + - Documentation +--- + +# Backup and Recovery +## Purpose +Find deployments for backup and recovery. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- Kopia + +## Follow the Subject +[Backup and Recovery](<../../reference/Backup and Recovery/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/reference/infrastructure/networking/Docker Networking/Docker Networking.md b/deployments/Containers/Docker/Create the Docker Network.md similarity index 52% rename from reference/infrastructure/networking/Docker Networking/Docker Networking.md rename to deployments/Containers/Docker/Create the Docker Network.md index bd99122..970c605 100644 --- a/reference/infrastructure/networking/Docker Networking/Docker Networking.md +++ b/deployments/Containers/Docker/Create the Docker Network.md @@ -4,11 +4,18 @@ tags: - Networking --- +## Purpose +This document records the procedure for create the docker network. Follow the environment assumptions and commands below. + ### Configure Docker Network We want to use a dedicated subnet / network specifically for containers, so they don't trample over the **SERVER** and **LAN** networks. If you are unsure of the name of the network adapter, in this case `eth0`, just type `ipaddr` in the terminal to list the network interfaces to locate it. -``` + +```text docker network create -d macvlan --subnet=192.168.5.0/24 --gateway=192.168.5.1 -o parent=eth0 docker_network ``` !!! note - Be sure to replace `eth0` with the correct interface name using `ip addr` in the terminal. e.g. It may appear as something else like `ens18`, etc. If the interface doesn't exist, Docker will produce an error complaining about it. \ No newline at end of file + Be sure to replace `eth0` with the correct interface name using `ip addr` in the terminal. e.g. It may appear as something else like `ens18`, etc. If the interface doesn't exist, Docker will produce an error complaining about it. + +## Related Documentation +- [Related Containers Documentation](<../../../reference/Containers/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/platforms/containerization/Docker/Deploy Portainer.md b/deployments/Containers/Docker/Deploy Portainer.md similarity index 81% rename from deployments/platforms/containerization/Docker/Deploy Portainer.md rename to deployments/Containers/Docker/Deploy Portainer.md index be59d4e..fd8f816 100644 --- a/deployments/platforms/containerization/Docker/Deploy Portainer.md +++ b/deployments/Containers/Docker/Deploy Portainer.md @@ -5,19 +5,22 @@ tags: - Containerization --- +## Purpose +This document records the procedure for deploy portainer. Follow the environment assumptions and commands below. + ### Update The Package Manager We need to update the server before installing Docker === "Ubuntu Server" - ``` sh + ```sh sudo apt update sudo apt upgrade -y ``` - + === "Rocky Linux" - ``` sh + ```sh sudo dnf check-update ``` @@ -25,7 +28,8 @@ We need to update the server before installing Docker Install Docker then deploy Portainer Convenience Script: -``` + +```text curl -fsSL https://get.docker.com | sudo sh dockerd-rootless-setuptool.sh install ``` @@ -34,30 +38,32 @@ Alternative Methods: === "Ubuntu Server" - ``` sh + ```sh sudo apt install docker.io -y docker run -d -p 8000:8000 -p 9443:9443 --name portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /srv/containers/portainer:/data portainer/portainer-ee:latest # (1) ``` - + 1. Be sure to set the `-v /srv/containers/portainer:/data` value to a safe place that gets backed up regularily. === "Rocky Linux" - ``` sh + ```sh sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo sudo dnf install -y docker-ce docker-ce-cli containerd.io sudo systemctl enable docker --now docker run -d -p 8000:8000 -p 9443:9443 --name portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /srv/containers/portainer:/data portainer/portainer-ee:latest # (2) ``` - + 1. This is needed to ensure that docker starts automatically every time the server is turned on. 2. Be sure to set the `-v /srv/containers/portainer:/data` value to a safe place that gets backed up regularily. ### Configure Docker Network -I highly recomment setting up a [Dedicated Docker MACVLAN Network](../../../../reference/infrastructure/networking/docker-networking/docker-networking.md). You can use it to keep your containers on their own subnet. +I highly recomment setting up a [Dedicated Docker MACVLAN Network](). You can use it to keep your containers on their own subnet. ### Access Portainer WebUI You will be able to access the Portainer WebUI at the following address: `https://:9443` !!! warning You need to be quick, as there is a timeout period where you wont be able to onboard / provision Portainer and will be forced to restart it's container. If this happens, you can find the container using `sudo docker container ls` proceeded by `sudo docker restart `. +## Related Documentation +- [Related Containers Documentation](<../../../reference/Containers/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/Containers/Docker/Git Repo Updater.md b/deployments/Containers/Docker/Git Repo Updater.md new file mode 100644 index 0000000..8d6aedc --- /dev/null +++ b/deployments/Containers/Docker/Git Repo Updater.md @@ -0,0 +1,69 @@ +--- +tags: + - Containers + - Docker + - Containerization +--- + +## Purpose +Docker container running Alpine Linux that automates and improves upon much of the script mentioned in the [Git Repo Updater](<../../../scripts/Automation/Gitea/Git Repo Updater.md>) document. It offers the additional benefits of checking for updates every 5 seconds instead of every 60 seconds. It also accepts environment variables to provide credentials and notification settings, and can have an infinite number of monitored repositories. + +!!! info "Earlier Configuration Delivery Approach" + The Gitea runners blog records replacing this polling approach for the documented configuration-delivery use case. The configuration-delivery reference links both runner implementations and this retained watcher deployment. + +### Deployment +You can find the current up-to-date Gitea repository that includes the `docker-compose.yml` and `.env` files that you need to deploy everything [here](https://git.bunny-lab.io/container-registry/-/packages/container/git-repo-updater/latest) + +```yaml title="docker-compose.yml" +version: '3.3' +services: + git-repo-updater: + privileged: true + container_name: git-repo-updater + env_file: + - stack.env + image: git.bunny-lab.io/container-registry/git-repo-updater:latest + volumes: + - /srv/containers:/srv/containers + - /srv/containers/git-repo-updater/Repo_Cache:/root/Repo_Cache + restart: always +``` + +```ini title=".env" +# Gitea Credentials +GIT_USERNAME=nicole.rappe +GIT_PASSWORD=USE-AN-APP-PASSWORD + +# NTFY Push Notification Server URL +NTFY_URL=https://ntfy.cyberstrawberry.net/git-repo-updater + +# Repository/Destination Pairs (Add as Many as Needed) +REPO_01="https://${GIT_USERNAME}:${GIT_PASSWORD}@git.bunny-lab.io/bunny-lab/docs.git,/srv/containers/material-mkdocs/docs/docs" +REPO_02="https://${GIT_USERNAME}:${GIT_PASSWORD}@git.bunny-lab.io/GitOps/servers.bunny-lab.io.git,/srv/containers/homepage-docker" +``` + +### Build / Development +If you want to learn how the container was assembled, the related build files are located [here](https://git.cyberstrawberry.net/container-registry/git-repo-updater) + +```dockerfile title="Dockerfile" +# Use Alpine as the base image of the container +FROM alpine:latest + +# Install necessary packages +RUN apk --no-cache add git curl rsync + +# Add script +COPY repo_watcher.sh /repo_watcher.sh +RUN chmod +x /repo_watcher.sh + +#Create Directory to store Repositories +RUN mkdir -p /root/Repo_Cache + +# Start script (Alpine uses /bin/sh instead of /bin/bash) +CMD ["/bin/sh", "-c", "/repo_watcher.sh"] +``` + +Save the complete [Git Repo Updater script](<../../../scripts/Automation/Gitea/Git Repo Updater.md>) as `repo_watcher.sh` next to the Dockerfile before building the image. + +## Related Documentation +- [Related Containers Documentation](<../../../reference/Containers/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/platforms/containerization/kubernetes/deployment/k8s.md b/deployments/Containers/Kubernetes/K8s.md similarity index 93% rename from deployments/platforms/containerization/kubernetes/deployment/k8s.md rename to deployments/Containers/Kubernetes/K8s.md index 0103b72..fe700a8 100644 --- a/deployments/platforms/containerization/kubernetes/deployment/k8s.md +++ b/deployments/Containers/Kubernetes/K8s.md @@ -4,11 +4,11 @@ tags: - Containerization --- -# Deploy Generic Kubernetes +## Purpose The instructions outlined below assume you are deploying the environment using Ansible Playbooks either via Ansible's CLI or AWX. ### Deploy K8S User -```jsx title="01-deploy-k8s-user.yml" +```yaml title="01-deploy-k8s-user.yml" - hosts: 'controller-nodes, worker-nodes' become: yes @@ -29,7 +29,7 @@ The instructions outlined below assume you are deploying the environment using A ``` ### Install K8S -```jsx title="02-install-k8s.yml" +```yaml title="02-install-k8s.yml" --- - hosts: "controller-nodes, worker-nodes" remote_user: nicole @@ -110,7 +110,7 @@ The instructions outlined below assume you are deploying the environment using A ``` ### Configure ControlPlanes -```jsx title="03-configure-controllers.yml" +```yaml title="03-configure-controllers.yml" - hosts: controller-nodes become: yes @@ -159,7 +159,7 @@ The instructions outlined below assume you are deploying the environment using A ``` ### Join Worker Node(s) -```jsx title="04-join-worker-nodes.yml" +```yaml title="04-join-worker-nodes.yml" - hosts: worker-nodes become: yes gather_facts: yes @@ -179,7 +179,7 @@ The instructions outlined below assume you are deploying the environment using A ``` ### Host Inventory File Template -```jsx title="hosts" +```text title="hosts" [controller-nodes] k8s-ctrlr-01 ansible_host=192.168.3.6 ansible_user=nicole @@ -191,3 +191,6 @@ k8s-node-02 ansible_host=192.168.3.5 ansible_user=nicole ansible_become_user=root ansible_become_method=sudo ``` + +## Related Documentation +- [Related Containers Documentation](<../../../reference/Containers/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/platforms/containerization/kubernetes/deployment/Rancher RKE2.md b/deployments/Containers/Kubernetes/Rancher RKE2.md similarity index 95% rename from deployments/platforms/containerization/kubernetes/deployment/Rancher RKE2.md rename to deployments/Containers/Kubernetes/Rancher RKE2.md index 763aaa6..397caaa 100644 --- a/deployments/platforms/containerization/kubernetes/deployment/Rancher RKE2.md +++ b/deployments/Containers/Kubernetes/Rancher RKE2.md @@ -6,7 +6,7 @@ tags: - Containerization --- -# Deploy RKE2 Cluster +## Purpose Deploying a Rancher RKE2 Cluster is fairly straightforward. Just run the commands in-order and pay attention to which steps apply to all machines in the cluster, the controlplanes, and the workers. !!! note "Prerequisites" @@ -17,7 +17,8 @@ Assume all commands are running as root moving forward. (e.g. `sudo su`) ### Run Updates You will need to run these commands on every server that participates in the cluster then perform a reboot of the server **PRIOR** to moving onto the next section. -``` sh + +```sh apt update && apt upgrade -y apt install nfs-common iptables nano htop -y echo "Adding 15 Second Delay to Ensure Previous Commands finish running" @@ -25,23 +26,26 @@ sleep 15 apt autoremove -y reboot ``` + !!! tip If this is a virtual machine, now would be the best time to take a checkpoint / snapshot of the VM before moving forward, in case you need to perform rollbacks of the server(s) if you accidentally misconfigure something. + ## Initial ControlPlane Node When you are starting a brand new cluster, you need to create what is referred to as the "Initial ControlPlane". This node is responsible for bootstrapping the entire cluster together in the beginning, and will eventually assist in handling container workloads and orchestrating operations in the cluster. !!! warning You only want to follow the instructions for the **initial** controlplane once. Running it on another machine to create additional controlplanes will cause the cluster to try to set up two different clusters, wrecking havok. Instead, follow the instructions in the next section to add redundant controlplanes. ### Download the Run Server Deployment Script -``` +```text curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE=server sh - ``` -### Enable & Configure Services -``` sh + +### Enable and Configure Services +```sh # Start and Enable the Kubernetes Service systemctl enable --now rke2-server.service -# Symlink the Kubectl Management Command +# Symlink the Kubectl Management Command ln -s $(find /var/lib/rancher/rke2/data/ -name kubectl) /usr/local/bin/kubectl # Temporarily Export the Kubeconfig to manage the cluster from CLI during initial deployment. @@ -59,15 +63,16 @@ kubectl get node When the cluster is ready, you should see something like this when you run `kubectl get node` This may be a good point to step away for 5 minutes, get a cup of coffee, and come back so it has a little extra time to be fully ready before moving on. - ``` + + ```text root@awx:/home/nicole# kubectl get node NAME STATUS ROLES AGE VERSION awx Ready control-plane,etcd,master 3m21s v1.26.12+rke2r1 ``` ### Install Helm, Rancher, CertManager, Jetstack, Rancher, and Longhorn -``` sh -# Install Helm +```sh +# Install Helm curl -L https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-4 | bash # Install Necessary Helm Repositories @@ -94,7 +99,8 @@ helm upgrade -i longhorn longhorn/longhorn --namespace longhorn-system --create- If you want to keep an eye on the deployment progress, you need to run the following command: `KUBECONFIG=/etc/rancher/rke2/rke2.yaml kubectl get pods --all-namespaces` The output should look like how it does below: - ``` + + ```text NAMESPACE NAME READY STATUS RESTARTS AGE cattle-fleet-system fleet-controller-59cdb866d7-94r2q 1/1 Running 0 4m31s cattle-fleet-system gitjob-f497866f8-t726l 1/1 Running 0 4m31s @@ -159,23 +165,25 @@ At this point, you can log into the webUI at https://rke2-cluster.bunny-lab.io u ### Rebooting the ControlNode If you ever find yourself needing to reboot the ControlNode, and need to run kubectl CLI commands, you will need to run the command below to import the cluster credentials upon every reboot. Reboots should take much less time to get the cluster ready again as compared to the original deployments. -``` + +```sh export KUBECONFIG=/etc/rancher/rke2/rke2.yaml ``` ## Create Additional ControlPlane Node(s) This is the part where you can add additional controlplane nodes to add additional redundancy to the RKE2 Cluster. This is important for high-availability environments. -### Download the Server Deployment Script -``` sh +### Download the Server Deployment Script +```sh curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE=server sh - ``` + ### Configure and Connect to Existing/Initial ControlPlane Node -``` sh -# Symlink the Kubectl Management Command +```sh +# Symlink the Kubectl Management Command ln -s $(find /var/lib/rancher/rke2/data/ -name kubectl) /usr/local/bin/kubectl -# Manually Create a Rancher-Kubernetes-Specific Config File +# Manually Create a Rancher-Kubernetes-Specific Config File mkdir -p /etc/rancher/rke2/ # Inject IP of Initial ControlPlane Node into Config File @@ -183,11 +191,12 @@ echo "server: https://192.168.3.69:9345" > /etc/rancher/rke2/config.yaml # Inject the Initial ControlPlane Node trust token into the config file # You can get the token by running the following command on the first node in the cluster: `cat /var/lib/rancher/rke2/server/node-token` -echo "token: K10aa0632863da4ae4e2ccede0ca6a179f510a0eee0d6d6eb53dca96050048f055e::server:3b130ceebfbb7ed851cd990fe55e6f3a" >> /etc/rancher/rke2/config.yaml +echo "token: " >> /etc/rancher/rke2/config.yaml # Start and Enable the Kubernetes Service systemctl enable --now rke2-server.service ``` + !!! note Be sure to change the IP address of the initial controlplane node provided in the example above to match your environment. @@ -195,20 +204,21 @@ systemctl enable --now rke2-server.service Worker nodes are the bread-and-butter of a Kubernetes cluster. They handle running container workloads, and acting as storage for the cluster (this can be configured to varying degrees based on your needs). ### Download the Server Worker Script -``` sh +```sh curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE=agent sh - ``` + ### Configure and Connect to RKE2 Cluster -``` sh -# Manually Create a Rancher-Kubernetes-Specific Config File +```sh +# Manually Create a Rancher-Kubernetes-Specific Config File mkdir -p /etc/rancher/rke2/ -# Inject IP of Initial ControlPlane Node into Config File +# Inject IP of Initial ControlPlane Node into Config File echo "server: https://192.168.3.21:9345" > /etc/rancher/rke2/config.yaml -# Inject the Initial ControlPlane Node trust token into the config file +# Inject the Initial ControlPlane Node trust token into the config file # You can get the token by running the following command on the first node in the cluster: `cat /var/lib/rancher/rke2/server/node-token` -echo "token: K10aa0632863da4ae4e2ccede0ca6a179f510a0eee0d6d6eb53dca96050048f055e::server:3b130ceebfbb7ed851cd990fe55e6f3a" >> /etc/rancher/rke2/config.yaml +echo "token: " >> /etc/rancher/rke2/config.yaml # Start and Enable the Kubernetes Service** systemctl enable --now rke2-agent.service @@ -219,8 +229,11 @@ You will need to set up some kind of DNS server record to point the FQDN of the Once you have added the DNS record, you should be able to access the login page for the Rancher RKE2 Kubernetes cluster. Use the `bootstrapPassword` mentioned previously to log in, then change it immediately from the user management area of Rancher. -| TYPE OF ACCESS | FQDN | IP ADDRESS | +| TYPE OF ACCESS | FQDN | IP ADDRESS | | -------------- | ------------------------------------- | ------------ | | HOST FILE | rke2-cluster.bunny-lab.io | 192.168.3.69 | | REVERSE PROXY | http://rke2-cluster.bunny-lab.io:80 | 192.168.5.29 | -| DNS RECORD | A Record: rke2-cluster.bunny-lab.io | 192.168.3.69 | \ No newline at end of file +| DNS RECORD | A Record: rke2-cluster.bunny-lab.io | 192.168.3.69 | + +## Related Documentation +- [Related Containers Documentation](<../../../reference/Containers/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/Containers/index.md b/deployments/Containers/index.md new file mode 100644 index 0000000..2f2eaa5 --- /dev/null +++ b/deployments/Containers/index.md @@ -0,0 +1,17 @@ +--- +tags: + - Containers + - Deployments + - Documentation +--- + +# Containers +## Purpose +Find deployments for containers. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- Docker +- Kubernetes + +## Follow the Subject +[Containers](<../../reference/Containers/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/deployments/services/authentication/Active Directory/Certificate Services.md b/deployments/Identity and Certificates/Active Directory/Certificate Services.md similarity index 75% rename from deployments/services/authentication/Active Directory/Certificate Services.md rename to deployments/Identity and Certificates/Active Directory/Certificate Services.md index 612f4ac..08b1df9 100644 --- a/deployments/services/authentication/Active Directory/Certificate Services.md +++ b/deployments/Identity and Certificates/Active Directory/Certificate Services.md @@ -6,22 +6,22 @@ tags: --- ## Purpose -This document outlines the Microsoft-recommended best practices for deploying a secure, internal-use-only, two-tier Public Key Infrastructure (PKI) using Windows Server 2022 or newer. The PKI supports securing S/MIME email, 802.1X Wi-Fi with NPS, and LDAP over SSL (LDAPS). +This document outlines the Microsoft-recommended best practices for deploying a secure, internal-use-only, two-tier Public Key Infrastructure (PKI) using Windows Server 2022 or newer. The PKI supports securing S/MIME email, 802.1X Wi-Fi with NPS, and LDAP over SSL (LDAPS). !!! abstract "CA Deployment Breakdown" - The environment will consist of at least 2 virtual machines. For the purposes of this document they will be named `LAB-CA-01` and `LAB-CA-02`. This stands for "*Lab Certificate Authority [01|02]*". In a two-tier hierarchy, an offline (*you intentionally keep this VM offline*) Root CA signs a single "*Subordinate*" Enterprise CA certificate. The Subordinate CA is domain-joined and handles all certificate requests. Clients trust the PKI via Group Policy and Active Directory integration. + The environment will consist of at least 2 virtual machines. For the purposes of this document they will be named `LAB-CA-01` and `LAB-CA-02`. This stands for "*Lab Certificate Authority [01|02]*". In a two-tier hierarchy, an offline (*you intentionally keep this VM offline*) Root CA signs a single "*Subordinate*" Enterprise CA certificate. The Subordinate CA is domain-joined and handles all certificate requests. Clients trust the PKI via Group Policy and Active Directory integration. In this case, `LAB-CA-01` is the Root CA, while `LAB-CA-02` is the Intermediary/Subordinate CA. You can add more than one subordinate CA if you desire more redundancy in your environment. Making them operate together is generally automatic and does not require manual intervention. !!! warning "Critical PKI Revocation Requirement" - CRL Distribution Points (CDP) and Authority Information Access (AIA) are required for this deployment, even if the immediate use case is only LDAPS. Certificate chain validation and revocation checking still occur for CA certificates and issued certificates. If CDP/AIA is missing, invalid, or unreachable, the Subordinate CA service may fail to start with revocation-related errors such as `0x80092013 (CRYPT_E_REVOCATION_OFFLINE)`. + CRL Distribution Points (CDP) and Authority Information Access (AIA) are required for this deployment, even if the immediate use case is only LDAPS. Certificate chain validation and revocation checking still occur for CA certificates and issued certificates. If CDP/AIA is missing, invalid, or unreachable, the Subordinate CA service may fail to start with revocation-related errors such as `0x80092013 (CRYPT_E_REVOCATION_OFFLINE)`. !!! note "Certificate Authority Server Provisioning Assumptions" - - OS = Windows Server 2022/2025 bare-metal or as a VM - - You should give it at least 4GB of RAM. - - [Change the edition of Windows Server from "**Evaluation**" to "**Standard**" via DISM](../../../../workflows/operations/windows/change-windows-edition.md) - - Ensure the server is fully updated - - [Ensure the server is activated](../../../../workflows/operations/windows/change-windows-edition.md#force-activation-edition-switcher) + - OS = Windows Server 2022/2025 bare-metal or as a VM + - You should give it at least 4GB of RAM. + - [Change the edition of Windows Server from "**Evaluation**" to "**Standard**" via DISM](<../../../workflows/Windows and Linux/Windows/Change Windows Edition.md>) + - Ensure the server is fully updated + - [Ensure the server is activated](<../../../workflows/Windows and Linux/Windows/Change Windows Edition.md#force-activation-edition-switcher>) - Ensure the timezone is correctly configured - Ensure the hostname is correctly configured @@ -33,11 +33,11 @@ This document outlines the Microsoft-recommended best practices for deploying a - `pki.bunny-lab.io` - This DNS name should resolve to the server hosting the PKI publication directory. In this deployment, the HTTP distribution point is hosted on `LAB-CA-02`. + This DNS name should resolve to the server hosting the PKI publication directory. In this deployment, the HTTP distribution point is hosted on `LAB-CA-02`. - This indirection is intentional. Certificate CDP/AIA URLs are embedded into issued certificates, so they should point to a stable DNS name rather than directly coupling clients to a CA hostname. + This indirection is intentional. Certificate CDP/AIA URLs are embedded into issued certificates, so they should point to a stable DNS name rather than directly coupling clients to a CA hostname. -## Offline (Non-Domain-Joined) Root CA `LAB-CA-01` +## Offline (Non-Domain-Joined) Root CA `LAB-CA-01` ### Role Deployment This is the initial deployment of the root certificate authority, the settings here should be double and triple checked before proceeding through each step. @@ -54,7 +54,7 @@ This is the initial deployment of the root certificate authority, the settings h - `Certification Authority Web Enrollment` - When prompted to confirm multiple times, click the "**Add Features**" button - Ensure the "**Include management tools (if applicable)**" checkbox is checked. - - The Root CA still requires properly configured CDP and AIA publication settings. These are configured after the CA role is installed and before the Subordinate CA certificate is issued. + - The Root CA still requires properly configured CDP and AIA publication settings. These are configured after the CA role is installed and before the Subordinate CA certificate is issued. - Click "**Next**" > "**Next**" > "**Next**" > "**Install**" - Restart the Server @@ -68,7 +68,7 @@ We have a few things we need to configure within the CA to make it ready to hand - Check the "**Root CA** radio box then click "**Next**" - Check the "**Create a new private key**" radio box then click "**Next**" - Click the dropdown menu for "**Select a crypotographic provider**" and ensure that "**RSA#Microsoft Software Key Storage Provider**" is selected - - *Microsoft Software Key Storage Provider (KSP) is the latest, most flexible provider designed to work with the Cryptography Next Generation (CNG) APIs. It offers better support for modern algorithms and improved security management (such as support for key attestation, better hardware integration, and improved key protection mechanisms).* + - *Microsoft Software Key Storage Provider (KSP) is the latest, most flexible provider designed to work with the Cryptography Next Generation (CNG) APIs. It offers better support for modern algorithms and improved security management (such as support for key attestation, better hardware integration, and improved key protection mechanisms).* - Set the key length to `4096` - Set the hash algorithm to `SHA256` - Click "**Next**" @@ -86,7 +86,7 @@ You will see a finalization screen confirming everything we have configured, it | Cryptographic provider | RSA#Microsoft Software Key Storage Provider | | Hash Algorithm | SHA256 | | Key Length | 4096 | -| Allow Administrator Interaction | Disabled | +| Allow Administrator Interaction | Disabled | | Certificate Validity Period | `<10 Years from Today>` | | Distinguished Name | CN=BunnyLab-RootCA,O=Bunny Lab,C=US | | Certificate Database Location | C:\Windows\system32\CertLog | @@ -96,12 +96,12 @@ You will see a finalization screen confirming everything we have configured, it If everything went well, you will see that the "**Certificate Authority**" and "**Certification Authority Web Enrollment**" both have a status of "**Configuration succeeded**". At this point, you can click the "**Close**" button to conclude the Root CA configuration. ### Configure Root CA CDP and AIA Publication URLs -Before the Root CA issues the Subordinate CA certificate, configure CDP and AIA publication URLs. These settings control what revocation and CA certificate locations are embedded into certificates issued by the Root CA. +Before the Root CA issues the Subordinate CA certificate, configure CDP and AIA publication URLs. These settings control what revocation and CA certificate locations are embedded into certificates issued by the Root CA. !!! warning "PowerShell Syntax" - The following commands are written for PowerShell. Use single percent signs such as `%3%8%9.crl`. + The following commands are written for PowerShell. Use single percent signs such as `%3%8%9.crl`. - If using `cmd.exe`, percent escaping may differ. Do not paste caret-based (`^`) line continuations into PowerShell. + If using `cmd.exe`, percent escaping may differ. Do not paste caret-based (`^`) line continuations into PowerShell. On `LAB-CA-01`, run: @@ -250,7 +250,7 @@ Invoke-WebRequest http://pki.bunny-lab.io/pki/LAB-CA-01_BunnyLab-RootCA.crt !!! note "Root CA Offline Model" After the Root CA files are published to the HTTP distribution point, `LAB-CA-01` can be taken offline again until the next CRL publication, CA renewal, or Subordinate CA certificate operation is required. -## Online (Domain-Joined) Subordinate/Intermediary CA `LAB-CA-02` +## Online (Domain-Joined) Subordinate/Intermediary CA `LAB-CA-02` ### Role Deployment Now that we have set up the root certificate authority, we can focus on setting up the subordinate CA. @@ -267,7 +267,7 @@ Now that we have set up the root certificate authority, we can focus on setting - Check the "**Subordinate CA**" radio box then click "**Next**" - Check the "**Create a new private key**" radio box then click "**Next**" - Click the dropdown menu for "**Select a crypotographic provider**" and ensure that "**RSA#Microsoft Software Key Storage Provider**" is selected - - *Microsoft Software Key Storage Provider (KSP) is the latest, most flexible provider designed to work with the Cryptography Next Generation (CNG) APIs. It offers better support for modern algorithms and improved security management (such as support for key attestation, better hardware integration, and improved key protection mechanisms).* + - *Microsoft Software Key Storage Provider (KSP) is the latest, most flexible provider designed to work with the Cryptography Next Generation (CNG) APIs. It offers better support for modern algorithms and improved security management (such as support for key attestation, better hardware integration, and improved key protection mechanisms).* - Set the key length to `4096` - Set the hash algorithm to `SHA256` - Click "**Next**" @@ -288,7 +288,7 @@ You will see a finalization screen confirming everything we have configured, it | Cryptographic provider | RSA#Microsoft Software Key Storage Provider | | Hash Algorithm | SHA256 | | Key Length | 4096 | -| Allow Administrator Interaction | Disabled | +| Allow Administrator Interaction | Disabled | | Certificate Validity Period | Determined by the parent CA | | Distinguished Name | CN=BunnyLab-SubordinateCA-01,DC=bunny-lab,DC=io | | Offline Request File Location | `C:\LAB-CA-02.bunny-lab.io_bunny-lab-LAB-CA-02-CA.req` | @@ -319,7 +319,7 @@ At this point, we will need to focus on getting the certificate signing request - Browse to and select the subordinate CA’s .req file (e.g. `LAB-CA-02.bunny-lab.io_bunny-lab-LAB-CA-02-CA.req`) - Click on "**BunnyLab-RootCA > Pending Requests** - Right-click the request we just imported, and select "**All Tasks > Issue**" - - Click on ""**BunnyLab-RootCA > Issued Certificates**" + - Click on ""**BunnyLab-RootCA > Issued Certificates**" - Locate the new subordinate CA certificate, and double-click it. - Click the "**Details**" tab - Click the "**Copy to File**" button @@ -340,15 +340,19 @@ At this point, we will need to focus on getting the certificate signing request - Browse to the `RootCA.cer` file exported from `LAB-CA-01` - Place all certificates in the following store: "Trusted Root Certification Authorities" - Click "**Next**" and finish importing the Root CA certificate - - Do **not** rely on manually importing the Root CA CRL into a certificate store as the revocation strategy. Revocation should be validated through the HTTP CDP URL embedded in the issued certificates. + - Do **not** rely on manually importing the Root CA CRL into a certificate store as the revocation strategy. Revocation should be validated through the HTTP CDP URL embedded in the issued certificates. - Verify Root CA CRL retrieval from `LAB-CA-02`: + ```powershell Invoke-WebRequest http://pki.bunny-lab.io/pki/BunnyLab-RootCA.crl ``` + - Verify the Subordinate CA certificate chain and revocation status: + ```powershell certutil -verify -urlfetch LAB-CA-02-SubCA.cer ``` + - Right-click the CA node in the treeview on the left-hand sidebar (e.g. `BunnyLab-SubordinateCA-01`) - Click on "**All Tasks" > "Start Service**" - Verify that the CA status is now green (running). @@ -401,12 +405,12 @@ C:\inetpub\wwwroot\pki\ Validate HTTP access to the Subordinate CA CRL and certificate after copying them to IIS. !!! note "Subordinate CA Publication" - The Root CA signs the Subordinate CA certificate. The Subordinate CA signs domain/server/client certificates. Therefore, clients need access to both Root CA revocation data and Subordinate CA revocation data. + The Root CA signs the Subordinate CA certificate. The Subordinate CA signs domain/server/client certificates. Therefore, clients need access to both Root CA revocation data and Subordinate CA revocation data. ## Reissuing the Subordinate CA Certificate After CDP/AIA Corrections If the Root CA CDP/AIA settings were configured or corrected after the original Subordinate CA certificate was issued, the Subordinate CA certificate must be reissued. -This is required because CDP/AIA values are embedded into issued certificates at issuance time. Changing the Root CA registry values later does not retroactively fix already-issued Subordinate CA certificates. +This is required because CDP/AIA values are embedded into issued certificates at issuance time. Changing the Root CA registry values later does not retroactively fix already-issued Subordinate CA certificates. ### Generate a New Subordinate CA Renewal Request On `LAB-CA-02`, run: @@ -415,7 +419,7 @@ On `LAB-CA-02`, run: certutil -renewCert ReuseKeys ``` -If prompted to submit the request to an online parent CA, click **Cancel**. This is expected for an offline Root CA workflow. A new `.req` file will be generated locally. +If prompted to submit the request to an online parent CA, click **Cancel**. This is expected for an offline Root CA workflow. A new `.req` file will be generated locally. ### Submit the Renewal Request to the Root CA Copy the generated `.req` file to `LAB-CA-01`. @@ -479,7 +483,7 @@ The Certificate Auto-Enrollment Group Policy enables domain-joined devices (*com - Run a `gpupdate /force` on your domain controller(s) and give it a few minutes to pull down their new domain controller certificates !!! note "Reissuing Certificates After PKI Corrections" - If any machine certificates, domain controller certificates, terminal server certificates, NPS certificates, or other service certificates were issued before CDP/AIA was configured correctly, reissue them. Old certificates may continue to work in some paths, but revocation checking and chain validation may be inconsistent. + If any machine certificates, domain controller certificates, terminal server certificates, NPS certificates, or other service certificates were issued before CDP/AIA was configured correctly, reissue them. Old certificates may continue to work in some paths, but revocation checking and chain validation may be inconsistent. For auto-enrolled certificates, use: @@ -505,8 +509,8 @@ At this point, you need to check that there is a certificate installed within "* Lastly, we want to ensure that LDAPS is functioning. By default, once these certs are enrolled on the domain controller(s), LDAPS *should* just work out of the box. To verify this, you can run this command on any device on the same network as the domain controllers. If it comes back successful like in the following example output, then you are golden: ```powershell -PS C:\Users\nicole.rappe> Test-NetConnection LAB-DC-01.bunny-lab.io -Port 636 -ComputerName : LAB-DC-01.bunny-lab.io +PS C:\Users\nicole.rappe> Test-NetConnection LAB-DC-01.bunny-lab.io -Port 636 +ComputerName : LAB-DC-01.bunny-lab.io RemoteAddress : 192.168.3.25 RemotePort : 636 InterfaceAlias : Ethernet @@ -550,104 +554,11 @@ Run an equivalent check for the Subordinate CA CRL after confirming the exact ge C:\Windows\System32\CertSrv\CertEnroll\ ``` -## CRL Publishing and Maintenance -CRLs must be generated and published on a recurring basis. If a CRL expires, certificate validation may fail even if the CA services themselves are running. +## Maintain Certificate Revocation Lists +After deployment, use [Publish and Maintain Certificate Revocation Lists](<../../../workflows/Identity and Certificates/Certificates/Publish and Maintain Certificate Revocation Lists.md>) for recurring publication and monitoring. -### Root CA CRL Publishing -Because the Root CA is offline, periodically bring it online only long enough to generate a new CRL and copy it to the HTTP distribution point. +## Connect LDAPS Clients +Use [Export Certificates for LDAPS Clients](<../../../workflows/Identity and Certificates/Certificates/Export Certificates for LDAPS Clients.md>) when an application needs the certificate chain. -On `LAB-CA-01`: - -```powershell -certutil -crl -``` - -Copy the generated CRL from: - -```text -C:\Windows\System32\CertSrv\CertEnroll\ -``` - -to the IIS publication directory on `LAB-CA-02`: - -```text -C:\inetpub\wwwroot\pki\ -``` - -Validate: - -```powershell -Invoke-WebRequest http://pki.bunny-lab.io/pki/BunnyLab-RootCA.crl -``` - -### Subordinate CA CRL Publishing -On `LAB-CA-02`: - -```powershell -certutil -crl -``` - -Copy or confirm the Subordinate CA CRL exists in: - -```text -C:\inetpub\wwwroot\pki\ -``` - -Validate the URL from a domain-joined system. - -### Operational Monitoring -Monitor CRL expiration and publication. Certificate validation failures can occur if CRLs expire, even if certificates themselves have not expired. - -Recommended operational tasks: - -- Track Root CA CRL expiration. -- Track Subordinate CA CRL expiration. -- Verify HTTP CRL URLs after each publication. -- Keep the Root CA offline except during controlled maintenance windows. -- Document the expected CRL filenames generated in `C:\Windows\System32\CertSrv\CertEnroll\`. - -!!! abstract "Raw Unprocessed/Unimplemented Steps" - Publish CRLs regularly, configure overlap periods, and monitor expiration. Enable Delta CRLs on the Subordinate CA, but not on the Root. - Security Recommendations - - - Harden CA servers; limit access to PKI admins. - - Use BitLocker or HSM for key protection. - - Monitor issuance and renewals with audit logs and scripts. - -## Export the LDAPS Certificate for Third-Party Applications -Some applications do not automatically trust your internal PKI and require you to manually install the certificate used by your domain controllers for LDAPS. In most cases, you should export the issuing CA certificates rather than the individual domain controller certificate. Only export the domain controller certificate if the third-party application explicitly requires it. - -### Export the Root and Subordinate CA Certificates -The Root CA and Subordinate CA certificates establish trust for every domain controller certificate issued by your PKI. - -From any domain-joined system: - -* Launch `certlm.msc` - * Navigate to "**Trusted Root Certification Authorities > Certificates**" - * Locate your Root CA certificate - * Right-click the certificate and select "**All Tasks > Export...**" - * Select "**No, do not export the private key**" - * Export the certificate as either: - * `DER encoded binary X.509 (.CER)`, or - * `Base-64 encoded X.509 (.CER)` - * Navigate to "**Intermediate Certification Authorities > Certificates**" - * Locate your Subordinate CA certificate - * Repeat the export process - -Import both certificates into the trusted certificate store required by the third-party application. - -### Export a Domain Controller Certificate -If the application requires the LDAPS server certificate itself: - -* On the target domain controller, launch `certlm.msc` - * Navigate to "**Personal > Certificates**" - * Locate the certificate issued to the domain controller's FQDN that includes **Server Authentication** as an intended purpose - * If the certificate's intended purpose looks like `Client Authentication, Server Authentication, Smart Card Logon, KDC Authentication` this cert may be more versatile for you. - * Right-click the certificate and select "**All Tasks > Export...**" - * Select "**No, do not export the private key**" - * Export the certificate as either: - * `DER encoded binary X.509 (.CER)`, or - * `Base-64 encoded X.509 (.CER)` - -!!! warning "Do Not Export the Private Key" - Third-party LDAPS clients require only the public certificate. Do **not** export the certificate as a `.pfx` file or include the private key unless the vendor explicitly documents that requirement. +## Related Documentation +- [Related Identity and Certificates Documentation](<../../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/authentication/authelia.md b/deployments/Identity and Certificates/Authelia.md similarity index 52% rename from deployments/services/authentication/authelia.md rename to deployments/Identity and Certificates/Authelia.md index b4ddab5..a291a20 100644 --- a/deployments/services/authentication/authelia.md +++ b/deployments/Identity and Certificates/Authelia.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Authelia is an open-source authentication and authorization server and portal fulfilling the identity and access management (IAM) role of information security in providing multi-factor authentication and single sign-on (SSO) for your applications via a web portal. It acts as a companion for common reverse proxies. +## Purpose +Authelia is an open-source authentication and authorization server and portal fulfilling the identity and access management (IAM) role of information security in providing multi-factor authentication and single sign-on (SSO) for your applications via a web portal. It acts as a companion for common reverse proxies. ```yaml title="docker-compose.yml" services: @@ -47,6 +48,6 @@ networks: external: true ``` -```jsx title=".env" -Not Applicable -``` +## Related Documentation +- [Docker Network Prerequisite](<../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Identity and Certificates Documentation](<../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/authentication/authentik.md b/deployments/Identity and Certificates/Authentik.md similarity index 82% rename from deployments/services/authentication/authentik.md rename to deployments/Identity and Certificates/Authentik.md index 5314559..91a9960 100644 --- a/deployments/services/authentication/authentik.md +++ b/deployments/Identity and Certificates/Authentik.md @@ -5,10 +5,11 @@ tags: - Docker --- -!!! bug - The docker-compose version of the deployment appears bugged and has known issues, deployment via Kubernetes is required to stability and support. +## Purpose +Authentik is an open-source Identity Provider, focused on flexibility and versatility. With authentik, site administrators, application developers, and security engineers have a dependable and secure solution for authentication in almost any type of environment. There are robust recovery actions available for the users and applications, including user profile and password management. You can quickly edit, deactivate, or even impersonate a user profile, and set a new password for new users or reset an existing password. -**Purpose**: Authentik is an open-source Identity Provider, focused on flexibility and versatility. With authentik, site administrators, application developers, and security engineers have a dependable and secure solution for authentication in almost any type of environment. There are robust recovery actions available for the users and applications, including user profile and password management. You can quickly edit, deactivate, or even impersonate a user profile, and set a new password for new users or reset an existing password. +!!! bug + The docker-compose version of the deployment appears bugged and has known issues, deployment via Kubernetes is required to stability and support. This document is based on the [Official Docker-Compose Documentation](https://goauthentik.io/docs/installation/docker-compose). It is meant for testing / small-scale production deployments. @@ -142,7 +143,7 @@ AUTHENTIK_BOOTSTRAP_EMAIL=nicole.rappe@bunny-lab.io ``` !!! note "Generating Passwords" - Navigate to the online [PWGen Password Generator](https://pwgen.io/en/) to generate the passwords for `PG_PASS` (40 characters) and `AUTHENTIK_SECRET_KEY` (50 characters). + Navigate to the online [PWGen Password Generator](https://pwgen.io/en/) to generate the passwords for `PG_PASS` (40 characters) and `AUTHENTIK_SECRET_KEY` (50 characters). Because of a PostgreSQL limitation, only passwords up to 99 characters are supported See https://www.postgresql.org/message-id/09512C4F-8CB9-4021-B455-EF4C4F0D55A0@amazon.com @@ -155,7 +156,8 @@ To start the initial setup, navigate to https://192.168.5.4:9443/if/flow/initial ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. -``` yaml + +```yaml http: routers: PLACEHOLDER: @@ -172,4 +174,9 @@ http: servers: - url: http://PLACEHOLDER:80 passHostHeader: true -``` \ No newline at end of file +``` + +## Related Documentation +- [Docker Network Prerequisite](<../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Identity and Certificates Documentation](<../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/authentication/keycloak/deployment.md b/deployments/Identity and Certificates/Keycloak/Deploy Keycloak.md similarity index 91% rename from deployments/services/authentication/keycloak/deployment.md rename to deployments/Identity and Certificates/Keycloak/Deploy Keycloak.md index 2cc2fc9..1f555cb 100644 --- a/deployments/services/authentication/keycloak/deployment.md +++ b/deployments/Identity and Certificates/Keycloak/Deploy Keycloak.md @@ -5,14 +5,15 @@ tags: - Docker --- -**Purpose**: Keycloak is an open source identity and access management system for modern applications and services. +## Purpose +Keycloak is an open source identity and access management system for modern applications and services. - [Original Reference Compose File](https://github.com/JamesTurland/JimsGarage/blob/main/Keycloak/docker-compose.yaml) - [Original Reference Deployment Video](https://www.youtube.com/watch?v=6ye4lP9EA2Y) - [Theme Customization Documentation](https://www.baeldung.com/spring-keycloak-custom-themes) ## Keycloak Authentication Sequence -``` mermaid +```mermaid sequenceDiagram participant User participant Traefik as Traefik Reverse Proxy @@ -30,8 +31,8 @@ sequenceDiagram Services->>Traefik: Response back to Traefik Traefik->>User: Return service response ``` -## Docker Configuration +## Docker Configuration === "docker-compose.yml" ```yaml @@ -114,7 +115,7 @@ sequenceDiagram - subnet: 172.16.238.0/24 # Subnet for internal network ``` - + 1. This sets the timezone of the Keycloak server to your timezone. This is not really necessary according to the official documentation, however I just like to add it to all of my containers as a baseline environment variable to add 2. This assumes you are running Keycloak behind a reverse proxy, in my particular case, Traefik 3. Set this to the FQDN that you are expecting to reach the Keycloak server at behind your reverse proxy @@ -134,12 +135,13 @@ sequenceDiagram KEYCLOAK_ADMIN=admin KEYCLOAK_ADMIN_PASSWORD=SomethingSuperSecureToLoginAsAdmin # (2) ``` - + 1. This is used internally by Keycloak to interact with the PostgreSQL database server 2. This is used to log into the web admin portal at https://auth.bunny-lab.io ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -173,12 +175,13 @@ http: X-Forwarded-Port: "443" ``` -# Traefik Keycloak Middleware -At this point, we need to add the official Keycloak plugin to Traefik's main configuration. In this example, it will be assumed you need to configure this in Portainer/Docker Compose, and not via a static yml/toml file. Assume you follow the [Docker Compose based Traefik Deployment](../../edge/traefik.md). +## Traefik Keycloak Middleware +At this point, we need to add the official Keycloak plugin to Traefik's main configuration. In this example, it will be assumed you need to configure this in Portainer/Docker Compose, and not via a static yml/toml file. Assume you follow the [Docker Compose based Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>). ## Install Keycloak Plugin If you do not already have the following added to the end of your `command:` section of the docker-compose.yml file in Portainer, go ahead and add it: -``` yaml + +```yaml # Keycloak plugin configuration - "--experimental.plugins.keycloakopenid.moduleName=github.com/Gwojda/keycloakopenid" - "--experimental.plugins.keycloakopenid.version=v0.1.34" @@ -187,7 +190,7 @@ If you do not already have the following added to the end of your `command:` sec ## Add Middleware to Traefik Dynamic Configuration You will want to ensure the following exists in the dynamically-loaded config file folder, you can name the file whatever you want, but it will be a one-all middleware for any services you want to have communicating as a specific OAuth2 `Client ID`. For example, you might want to have some services exist in a particular realm of Keycloak, or to have different client rules apply to certain services. If this is the case, you can create multiple middlewares in this single yaml file, each handling a different service / realm. It can get pretty complicated if you want to handle a multi-tenant environment, such as one seen in an enterprise environment. -```jsx title="keycloak-middleware.yml" +```yaml title="keycloak-middleware.yml" http: middlewares: auth-bunny-lab-io: @@ -215,6 +218,7 @@ At this point, you are in the final stretch, you just need to add the middleware ``` A full example config file would look like the following: + ```yaml http: routers: @@ -236,3 +240,7 @@ http: passHostHeader: true ``` +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Identity and Certificates Documentation](<../../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/Security and Utility/Password Pusher.md b/deployments/Identity and Certificates/Password Pusher.md similarity index 77% rename from deployments/services/Security and Utility/Password Pusher.md rename to deployments/Identity and Certificates/Password Pusher.md index f2eec1a..151f143 100644 --- a/deployments/services/Security and Utility/Password Pusher.md +++ b/deployments/Identity and Certificates/Password Pusher.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: An application to securely communicate passwords over the web. Passwords automatically expire after a certain number of views and/or time has passed. Track who, what and when. +## Purpose +An application to securely communicate passwords over the web. Passwords automatically expire after a certain number of views and/or time has passed. Track who, what and when. ## Docker Configuration ```yaml title="docker-compose.yml" @@ -69,6 +70,7 @@ PWP__LOG_LEVEL=info ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -86,4 +88,9 @@ http: servers: - url: http://192.168.5.170:5100 passHostHeader: true -``` \ No newline at end of file +``` + +## Related Documentation +- [Docker Network Prerequisite](<../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Identity and Certificates Documentation](<../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/services/authentication/privacyidea.md b/deployments/Identity and Certificates/Privacyidea.md similarity index 90% rename from deployments/services/authentication/privacyidea.md rename to deployments/Identity and Certificates/Privacyidea.md index 4eb3750..c2172ac 100644 --- a/deployments/services/authentication/privacyidea.md +++ b/deployments/Identity and Certificates/Privacyidea.md @@ -4,14 +4,15 @@ tags: - Authentication --- -**Purpose**: privacyIDEA is a modular authentication system. Using privacyIDEA you can enhance your existing applications like local login, VPN, remote access, SSH connections, access to web sites or web portals with a second factor during authentication. +## Purpose +privacyIDEA is a modular authentication system. Using privacyIDEA you can enhance your existing applications like local login, VPN, remote access, SSH connections, access to web sites or web portals with a second factor during authentication. !!! info "Assumptions" It is assumed you have a provisioned virtual machine / physical machine, running Ubuntu Server 22.04 to deploy a privacyIDEA server. ## AWX Deployment ### Add Server to Inventory and Pull Inventory/Playbook Updates from Gitea -You need to target the new server using a template in AWX (preferrably). +You need to target the new server using a template in AWX (preferrably). - We will assume the FQDN of the server is `auth.bunny-lab.io` or just `auth` - Be sure to add the host into the [AWX Homelab Inventory File](https://git.bunny-lab.io/GitOps/awx.bunny-lab.io/src/branch/main/inventories/homelab.ini) @@ -46,7 +47,8 @@ Now we need to launch the template. Assuming all of the above was completed, we !!! success You will know if everything was successful if you see something that looks like the following: - ``` sh + + ```sh ok: [auth] TASK [Install wget and software-properties-common] ***************************** ok: [auth] @@ -60,13 +62,14 @@ Now we need to launch the template. Assuming all of the above was completed, we changed: [auth] TASK [Install PrivacyIDEA with Apache2] **************************************** changed: [auth] - PLAY RECAP *********************************************************************auth : ok=7 changed=5 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 + PLAY RECAP *********************************************************************auth : ok=7 changed=5 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 ``` ## Admin Access to WebUI ### Create a privacyIDEA Administrator Account You will need to use the CLI in the server in order to create the first administrative account. Run the following command and provide a password for the administrator account. -``` sh + +```sh sudo pi-manage admin add nicole.rappe -e nicole.rappe@bunny-lab.io ``` @@ -111,7 +114,7 @@ You will need to create several policies, you can make them all individual, or m ## Enrolling the First Token !!! bug "Push Notifications Broken" - Currently, the push notification system (e.g. Cisco DUO") is not behaving as-expected. For now, you can use other authentication methods for the tokens, such as HOTP (on-demand MFA codes) or TOTP (conventional time-based MFA codes). + Currently, the push notification system (e.g. Cisco DUO") is not behaving as-expected. For now, you can use other authentication methods for the tokens, such as HOTP (on-demand MFA codes) or TOTP (conventional time-based MFA codes). ### TOTP Token Navigate to "**Tokens > Enroll Token**" @@ -135,10 +138,13 @@ When you want to leverage MFA in an environment using the server, you need to ha - Click "**Next**" - Check the "**Agree**" checkbox, then click "**Next**" - Hostname: `auth.bunny-lab.io` -- Path: `/path/to/pi` +- Path: `/path/to/pi` - [x] Ignore Unknown CA Errors when Using SSL - [x] Ignore Invalid Common Name Errors when Using SSL - Click "**Next**" > "**Next**" > "**Next**" - Click "**Install**" then "**Finish**" -You can now log out and verify that the credential provider is displayed as an option, and can log in using your domain username, domain password, and TOTP that you configured in the privacyIDEA WebUI. \ No newline at end of file +You can now log out and verify that the credential provider is displayed as an option, and can log in using your domain username, domain password, and TOTP that you configured in the privacyIDEA WebUI. + +## Related Documentation +- [Related Identity and Certificates Documentation](<../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/Security and Utility/Vaultwarden.md b/deployments/Identity and Certificates/Vaultwarden.md similarity index 73% rename from deployments/services/Security and Utility/Vaultwarden.md rename to deployments/Identity and Certificates/Vaultwarden.md index 9c9b894..4e18c16 100644 --- a/deployments/services/Security and Utility/Vaultwarden.md +++ b/deployments/Identity and Certificates/Vaultwarden.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Unofficial Bitwarden compatible server written in Rust, formerly known as bitwarden_rs. +## Purpose +Unofficial Bitwarden compatible server written in Rust, formerly known as bitwarden_rs. ```yaml title="docker-compose.yml" --- @@ -41,14 +42,17 @@ networks: docker_network: external: true ``` + !!! warning "ADMIN_TOKEN" It is **CRITICAL** that you never share the `ADMIN_TOKEN` with anyone. It allows you to log into the instance at https://vault.example.com/admin to add users, delete users, make changes system wide, etc. ```yaml title=".env" Not Applicable ``` + ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -67,3 +71,8 @@ http: - url: http://192.168.5.15:80 passHostHeader: true ``` + +## Related Documentation +- [Docker Network Prerequisite](<../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Identity and Certificates Documentation](<../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/Identity and Certificates/index.md b/deployments/Identity and Certificates/index.md new file mode 100644 index 0000000..e63b908 --- /dev/null +++ b/deployments/Identity and Certificates/index.md @@ -0,0 +1,22 @@ +--- +tags: + - Identity and Certificates + - Deployments + - Documentation +--- + +# Identity and Certificates +## Purpose +Find deployments for identity and certificates. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- Active Directory +- Authelia +- Authentik +- Keycloak +- Password Pusher +- Privacyidea +- Vaultwarden + +## Follow the Subject +[Identity and Certificates](<../../reference/Identity and Certificates/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/deployments/Networking and Access/DNS/AdGuard Home.md b/deployments/Networking and Access/DNS/AdGuard Home.md new file mode 100644 index 0000000..13fb23e --- /dev/null +++ b/deployments/Networking and Access/DNS/AdGuard Home.md @@ -0,0 +1,38 @@ +--- +tags: + - AdGuard Home + - DNS + - Docker +--- + +## Purpose +AdGuard Home is a network-wide software for blocking ads & tracking. After you set it up, it will cover ALL your home devices, and you don’t need any client-side software for that. With the rise of Internet-Of-Things and connected devices, it becomes more and more important to be able to control your whole network. + +```yaml title="docker-compose.yml" +version: '3' + +services: + app: + image: adguard/adguardhome + ports: + - 3000:3000 + - 53:53 + - 80:80 + volumes: + - /srv/containers/adguard_home/workingdir:/opt/adguardhome/work + - /srv/containers/adguard_home/config:/opt/adguardhome/conf + restart: always + networks: + docker_network: + ipv4_address: 192.168.5.189 +networks: + default: + external: + name: docker_network + docker_network: + external: true +``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Networking and Access Documentation](<../../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/DNS/Pi Hole.md b/deployments/Networking and Access/DNS/Pi Hole.md similarity index 66% rename from deployments/services/DNS/Pi Hole.md rename to deployments/Networking and Access/DNS/Pi Hole.md index 841979b..f8c4de7 100644 --- a/deployments/services/DNS/Pi Hole.md +++ b/deployments/Networking and Access/DNS/Pi Hole.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Pi-hole is a Linux network-level advertisement and Internet tracker blocking application which acts as a DNS sinkhole and optionally a DHCP server, intended for use on a private network. +## Purpose +Pi-hole is a Linux network-level advertisement and Internet tracker blocking application which acts as a DNS sinkhole and optionally a DHCP server, intended for use on a private network. ```yaml title="docker-compose.yml" version: "3" @@ -46,3 +47,7 @@ networks: ```yaml title=".env" Not Applicable ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Networking and Access Documentation](<../../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/infrastructure/networking/vpn/netbird.md b/deployments/Networking and Access/NetBird/Deploy NetBird on Rocky Linux.md similarity index 89% rename from reference/infrastructure/networking/vpn/netbird.md rename to deployments/Networking and Access/NetBird/Deploy NetBird on Rocky Linux.md index 217b181..4f3a3bb 100644 --- a/reference/infrastructure/networking/vpn/netbird.md +++ b/deployments/Networking and Access/NetBird/Deploy NetBird on Rocky Linux.md @@ -23,7 +23,7 @@ You need to install a few things before we can begin with the deployment of Netb sudo dnf update -y sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo sudo dnf install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin jq -sudo systemctl enable docker --now +sudo systemctl enable docker --now # Configure normal user to have docker privileges sudo usermod -aG docker nicole @@ -49,4 +49,7 @@ If everything is working correctly, you can go make some coffee and come back. ```sh -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Networking and Access Documentation](<../../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/Remote Access/Apache Guacamole.md b/deployments/Networking and Access/Remote Access/Apache Guacamole.md similarity index 87% rename from deployments/services/Remote Access/Apache Guacamole.md rename to deployments/Networking and Access/Remote Access/Apache Guacamole.md index 25df8d3..f30fe4c 100644 --- a/deployments/services/Remote Access/Apache Guacamole.md +++ b/deployments/Networking and Access/Remote Access/Apache Guacamole.md @@ -4,7 +4,8 @@ tags: - Docker --- -**Purpose**: HTML5-based Remote Access Broker for SSH, RDP, and VNC. Useful for remote access into an environment. +## Purpose +HTML5-based Remote Access Broker for SSH, RDP, and VNC. Useful for remote access into an environment. ### Docker Compose Stack === "docker-compose.yml" @@ -45,7 +46,7 @@ tags: docker_network: external: true ``` - + 1. Enable this if you want multi-factor authentication enabled. Must be set BEFORE the container is initially deployed. Cannot be added retroactively. 2. Set to your own timezone. @@ -92,14 +93,14 @@ tags: docker_network: external: true ``` - + 1. You cannot enable TOTP / Multi-factor authentication if you have OpenID configured. This is just a known issue. 2. Set to your own timezone. ### Environment Variables === ".env" - ``` sh + ```sh N/A ``` @@ -113,10 +114,9 @@ tags: ``` ## Reverse Proxy Configuration - === "Traefik" - ``` yaml + ```yaml http: routers: apache-guacamole: @@ -134,7 +134,7 @@ tags: - url: http://192.168.5.43:8080 passHostHeader: true ``` - + === "NGINX" ```yaml @@ -153,4 +153,8 @@ tags: access_log off; } } - ``` \ No newline at end of file + ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Networking and Access Documentation](<../../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/Remote Access/Firefox.md b/deployments/Networking and Access/Remote Access/Firefox.md similarity index 85% rename from deployments/services/Remote Access/Firefox.md rename to deployments/Networking and Access/Remote Access/Firefox.md index bb2f705..babdda8 100644 --- a/deployments/services/Remote Access/Firefox.md +++ b/deployments/Networking and Access/Remote Access/Firefox.md @@ -4,10 +4,11 @@ tags: - Docker --- -**Purpose**: Sometimes you just want an instance of Firefox running on an Alpine Linux container, that has persistence (Extensions, bookmarks, history, etc) outside of the container (with bind-mapped folders). This is useful for a number of reasons, but insecure by default, so you have to protect it behind something like a [Keycloak Server](../authentication/keycloak/deployment.md) so it is not misused. +## Purpose +Sometimes you just want an instance of Firefox running on an Alpine Linux container, that has persistence (Extensions, bookmarks, history, etc) outside of the container (with bind-mapped folders). This is useful for a number of reasons, but insecure by default, so you have to protect it behind something like a [Keycloak Server](<../../Identity and Certificates/Keycloak/Deploy Keycloak.md>) so it is not misused. ## Keycloak Authentication Sequence -``` mermaid +```mermaid sequenceDiagram participant User participant Traefik as Traefik Reverse Proxy @@ -56,7 +57,7 @@ It is important, due to how this browser just allows anyone to access it, to loc These rules will drop all traffic by default, allow port 22, and restrict access to port 5800. -``` sh +```sh # Set the default zone to drop sudo firewall-cmd --set-default-zone=drop @@ -76,6 +77,7 @@ sudo firewall-cmd --reload ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -108,3 +110,6 @@ Due to the nature of how this is deployed, you need to make some additional conf - **Install Merge All Windows Extension**: At times, you may misclick somewhere in the Firefox environment causing Firefox to open a new instance / window losing all of your tabs, and because there is no window manager, there is no way to alt+tab or switch between the instances of Firefox, effectively breaking your current session forcing you to re-open tabs. With this extension, you can merge all of the windows, collapsing them into one window, resolving the issue. - **Configure New Tab behavior**: If a new tab opens in a new window, it will absolutely throw everything into disarray, that is why all hyperlinks will be forced to open in a new tab instead of a new window. You can do this by navigating to `about:config` and setting the variable `browser.link.open_newwindow.restriction` to a value of `0`. [Original Reference Documentation](https://support.mozilla.org/en-US/questions/1066799) +## Related Documentation +- [Related Networking and Access Documentation](<../../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/workflows/operations/Linux/Fedora Workstation/Install DE into Fedora Server.md b/deployments/Networking and Access/Remote Access/Install XFCE and RustDesk on Fedora Server.md similarity index 91% rename from workflows/operations/Linux/Fedora Workstation/Install DE into Fedora Server.md rename to deployments/Networking and Access/Remote Access/Install XFCE and RustDesk on Fedora Server.md index fd280c1..a00dc21 100644 --- a/workflows/operations/Linux/Fedora Workstation/Install DE into Fedora Server.md +++ b/deployments/Networking and Access/Remote Access/Install XFCE and RustDesk on Fedora Server.md @@ -9,16 +9,18 @@ tags: ## Purpose You may find that you need to install an XFCE desktop environment or something into Fedora Server, if this is the case, for installing something like Rustdesk remote access, you can follow the steps below. -### Install & Configure XFCE +### Install and Configure XFCE We need to install XFCE and configure it to be the default environment when the server turns on. + ```sh sudo dnf install @xfce-desktop-environment -y sudo systemctl set-default graphical.target sudo reboot ``` -#### Install Rustdesk: +#### Install Rustdesk We need to install Rustdesk into the server. + ```sh curl -L -o /tmp/rustdesk_installer.rpm https://github.com/rustdesk/rustdesk/releases/download/1.4.0/rustdesk-1.4.0-0.x86_64.rpm cd /tmp @@ -34,6 +36,7 @@ sudo yum install rustdesk_installer.rpm -y For Rustdesk specifically, we have to configure XFCE to automatically login via SDDM then immediately lock the computer once it's logged in, so the XFCE session is running, allowing Rustdesk to connect to it. **Create SDDM Config File**: + ```sh sudo mkdir -p /etc/sddm.conf.d/ sudo nano /etc/sddm.conf.d/autologin.conf @@ -71,6 +74,10 @@ Comment=Lock the screen on login ``` Lastly, test that everything is working by rebooting the server. + ```sh sudo reboot -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Networking and Access Documentation](<../../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Bash/Install XRDP.md b/deployments/Networking and Access/Remote Access/Install XRDP on Ubuntu.md similarity index 74% rename from scripts/Bash/Install XRDP.md rename to deployments/Networking and Access/Remote Access/Install XRDP on Ubuntu.md index 801e315..28ac7a7 100644 --- a/scripts/Bash/Install XRDP.md +++ b/deployments/Networking and Access/Remote Access/Install XRDP on Ubuntu.md @@ -6,10 +6,10 @@ tags: - Linux --- -**Purpose**: +## Purpose If you need to set up RDP access to a Linux environment, you will want to install XRDP. Once it is installed, you can leverage other tools such as Apache Guacamole to remotely connect to it. -``` +```sh # Install and Start XRDP Service sudo dnf install epel-release -y sudo dnf install xrdp -y @@ -23,4 +23,7 @@ sudo firewall-cmd --reload # XFCE4 Desktop Environment echo "startxfce4" > ~/.Xclients chmod +x ~/.Xclients -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Networking and Access Documentation](<../../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/edge/nginx.md b/deployments/Networking and Access/Reverse Proxies/Nginx.md similarity index 54% rename from deployments/services/edge/nginx.md rename to deployments/Networking and Access/Reverse Proxies/Nginx.md index 2c70cdb..d34a175 100644 --- a/deployments/services/edge/nginx.md +++ b/deployments/Networking and Access/Reverse Proxies/Nginx.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: NGINX is open source software for web serving, reverse proxying, caching, load balancing, media streaming, and more. +## Purpose +NGINX is open source software for web serving, reverse proxying, caching, load balancing, media streaming, and more. ```yaml title="docker-compose.yml" --- @@ -39,3 +40,7 @@ networks: ```yaml title=".env" Not Applicable ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Networking and Access Documentation](<../../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/edge/traefik.md b/deployments/Networking and Access/Reverse Proxies/Traefik.md similarity index 83% rename from deployments/services/edge/traefik.md rename to deployments/Networking and Access/Reverse Proxies/Traefik.md index 8fb0f2c..aeba14c 100644 --- a/deployments/services/edge/traefik.md +++ b/deployments/Networking and Access/Reverse Proxies/Traefik.md @@ -5,10 +5,11 @@ tags: - Docker --- -**Purpose**: A traefik reverse proxy is a server that sits between your network firewall and servers hosting various web services on your private network(s). Traefik automatically handles the creation of Let's Encrypt SSL certificates if you have a domain registrar that is supported by Traefik such as CloudFlare; by leveraging API keys, Traefik can automatically make the DNS records for Let's Encrypt's DNS "challenges" whenever you add a service behind the Traefik reverse proxy. +## Purpose +A traefik reverse proxy is a server that sits between your network firewall and servers hosting various web services on your private network(s). Traefik automatically handles the creation of Let's Encrypt SSL certificates if you have a domain registrar that is supported by Traefik such as CloudFlare; by leveraging API keys, Traefik can automatically make the DNS records for Let's Encrypt's DNS "challenges" whenever you add a service behind the Traefik reverse proxy. !!! info "Assumptions" - This Traefik deployment document assumes you have deployed [Portainer](../../platforms/containerization/docker/deploy-portainer.md) to either a Rocky Linux or Ubuntu Server environment. Other docker-compose friendly operating systems have not been tested, so your mileage may vary regarding successful deployment ouside of these two operating systems. + This Traefik deployment document assumes you have deployed [Portainer](<../../Containers/Docker/Deploy Portainer.md>) to either a Rocky Linux or Ubuntu Server environment. Other docker-compose friendly operating systems have not been tested, so your mileage may vary regarding successful deployment ouside of these two operating systems. Portainer makes deploying and updating Traefik so much easier than via a CLI. It's also much more intuitive. @@ -21,7 +22,7 @@ tags: !!! warning "Get DNS Registrar API Keys BEFORE DEPLOYMENT" When you are deploying this container, you have to be mindful to set valid data for the environment variables related to the DNS registrar. In this example, it is CloudFlare. - ```jsx title="Environment Variables" + ```text title="Environment Variables" CF_API_EMAIL=nicole.rappe@bunny-lab.io CF_API_KEY=REDACTED-CLOUDFLARE-DOMAIN-API-KEY ``` @@ -29,10 +30,10 @@ tags: If these are not set, Traefik will still work, but SSL certificates will not be issued from Let's Encrypt, and SSL traffic will be terminated using a self-signed Traefik-based certificate, which is only good for local non-production testing. If you plan on using HTTP-based challenges, you will need to make the following changes in the docker-compose.yml data: - + - Un-comment `"--certificatesresolvers.myresolver.acme.tlschallenge=true"` - Comment-out `"--certificatesresolvers.letsencrypt.acme.dnschallenge=true"` - - Comment-out `"--certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare"` + - Comment-out `"--certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare"` - Lastly, you need to ensure that port 80 on your firewall is opened to the IP of the Traefik Reverse Proxy to allow Let's Encrypt to do TLS-based challenges. ### Stack Deployment Information @@ -90,7 +91,7 @@ services: # Keycloak plugin configuration - "--experimental.plugins.keycloakopenid.moduleName=github.com/Gwojda/keycloakopenid" # Optional if you have Keycloak Deployed - "--experimental.plugins.keycloakopenid.version=v0.1.34" # Optional if you have Keycloak Deployed - + ports: - "80:80" - "443:443" @@ -130,7 +131,7 @@ LETSENCRYPT_EMAIL=nicole.rappe@bunny-lab.io ## Adding Servers / Services to Traefik Traefik operates in two ways, the first is labels, while the second are dynamic configuration files. We will go over each below. -### Docker-Compose Labels +### Docker-Compose Labels The first is that it reads "labels" from the docker-compose file of any deployed containers on the same host as Traefik. These labels typically look something like the following: ```yaml title="docker-compose.yml" @@ -145,7 +146,7 @@ The first is that it reads "labels" from the docker-compose file of any deployed By adding these labels to any container on the same server as Traefik, traefik will automatically "adopt" this service and route traffic to it as well as assign an SSL certificate to it from Let's Encrypt. The only downside is as mentioned above, if you are dealing with something that is not just a container, or maybe a container on a different physical server, you need to rely on dynamic configuration files, such as the one seen below. ### Dynamic Configuration Files -Dynamic configuration files exist under the Traefik container located at `/etc/traefik/dynamic`. Any `*.yml` files located in this folder will be hot-loaded anytime they are modified. This makes it convenient to leverage something such as the [Git Repo Updater](../../platforms/containerization/docker/custom-containers/git-repo-updater.md) container to leverage [Gitea](../devops/gitea.md) to push configuration files from Git into the production environment, saving yourself headache and enabling version control over every service behind the reverse proxy. +Dynamic configuration files exist under the Traefik container located at `/etc/traefik/dynamic`. Any `*.yml` files located in this folder will be hot-loaded anytime they are modified. This makes it convenient to leverage something such as the [Git Repo Updater](<../../Containers/Docker/Git Repo Updater.md>) container to leverage [Gitea](<../../automation/Gitea/Gitea.md>) to push configuration files from Git into the production environment, saving yourself headache and enabling version control over every service behind the reverse proxy. An example of a dynamic configuration file would look something like this: @@ -196,3 +197,7 @@ You can see the similarities between the labeling method and how you designate t passHostHeader: true ``` +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Gitea Configuration Delivery](<../../../reference/Automation/Gitea Configuration Delivery.md>) — Choose the delivery implementation for the dynamic configuration directory. +- [Related Networking and Access Documentation](<../../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/infrastructure/networking/Firewall and Routing/Sophos/VPN/Site to Site VPNs/IPSEC/Tunnel Creation.md b/deployments/Networking and Access/Sophos/Configure a Site-to-Site IPsec VPN.md similarity index 87% rename from reference/infrastructure/networking/Firewall and Routing/Sophos/VPN/Site to Site VPNs/IPSEC/Tunnel Creation.md rename to deployments/Networking and Access/Sophos/Configure a Site-to-Site IPsec VPN.md index 8e08d21..9059a2b 100644 --- a/reference/infrastructure/networking/Firewall and Routing/Sophos/VPN/Site to Site VPNs/IPSEC/Tunnel Creation.md +++ b/deployments/Networking and Access/Sophos/Configure a Site-to-Site IPsec VPN.md @@ -7,17 +7,17 @@ tags: - Routing --- -**Purpose**: You may have two Sophos XGS appliances (or a mixed configuration) and need to set up a site-to-site VPN tunnel between two remote locations. You can achieve this with a simple passphrase-based IPSec VPN tunnel. +## Purpose +You may have two Sophos XGS appliances (or a mixed configuration) and need to set up a site-to-site VPN tunnel between two remote locations. You can achieve this with a simple passphrase-based IPSec VPN tunnel. !!! info "Assumptions" - This documentation only provides instruction for Sophos XGS based devices. It does not account for third-party vendors or other manufactured hardware. If you need to set up a mixed VPN tunnel with a different brand of networking device, you need to do your best to match the settings on the tunnels manually. (e.g. Encryption Type, Phase Lifetimes, etc). + This documentation only provides instruction for Sophos XGS based devices. It does not account for third-party vendors or other manufactured hardware. If you need to set up a mixed VPN tunnel with a different brand of networking device, you need to do your best to match the settings on the tunnels manually. (e.g. Encryption Type, Phase Lifetimes, etc). ## Architecture - !!! tip "Best Practices - Initiators / Responders" If you have a hub-and-spoke network, where one location acts as a central authority (e.g. domain controllers, auth servers, identity providers, headquarters, etc), you will set up the central "hub" as a VPN responder on its side of the VPN tunnel, and all the remote "spoke" locations would behave as VPN initiators. -``` mermaid +```mermaid graph TB Responder((Responder
Headquarters)) Initiator1((Initiator
Remote Site 1)) @@ -40,7 +40,6 @@ You will need to access the firewall either directly on the local network at `ht Navigate to "**Configure > Site-to-Site VPN > Add**" ### General settings - | **Field** | **Value** | | :--- | :--- | | Name | ` to ` | @@ -49,19 +48,17 @@ Navigate to "**Configure > Site-to-Site VPN > Add**" | Gateway Type | `Initiate the Connection` / `Respond Only` (*See "Best Practices" Section*) | ### Encryption - | **Field** | **Value** | | :--- | :--- | | Encryption Profile | `Custom_IKEv2_Initiator` / `Custom_IKEv2_Responder` (*Based on the "Gateway Type"*) | -| Authentication Type | `Preshared Key / Passphrase` | +| Authentication Type | `Preshared Key / Passphrase` | ### Gateway Settings - | **Field** | **Value** | | :--- | :--- | | Listening Interface | `` (*Internal IP Address*) | | Gateway Address | `` | -| Local ID Type | `IP Address` (*Usually Optional*) | +| Local ID Type | `IP Address` (*Usually Optional*) | | Remote ID Type | `` (*Usually Optional*)| | Local Subnet | `` | | Remote Subnet | `` | @@ -82,14 +79,14 @@ Navigate to "**System > Profiles > IPSec Profiles > Custom_IKEv2_`/` | | Phase 2 Lifetime | *Default Value*: `14400` | `` | - + === "Responder Phase Lifetime Values" - | **Field** | **Value** | **Notes** | + | **Field** | **Value** | **Notes** | | :--- | :--- | :--- | | Phase 1 Lifetime | *Default Value + 300 Seconds*: `328800` | `` | | Phase 2 Lifetime | *Default Value + 300 Seconds*: `314400` | `` | @@ -103,3 +100,5 @@ You will need to repeat the steps on both firewalls, so one firewall is the init ## Connect the IPSec Tunnels Now you need to start the tunnel on the Initiator side first, then start the tunnel on the responder side. If both sides show green status indicators, the tunnel should be active. +## Related Documentation +- [Related Networking and Access Documentation](<../../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/infrastructure/networking/Controllers/UniFi Controller.md b/deployments/Networking and Access/UniFi/Deploy UniFi Controller with Docker.md similarity index 55% rename from reference/infrastructure/networking/Controllers/UniFi Controller.md rename to deployments/Networking and Access/UniFi/Deploy UniFi Controller with Docker.md index 9eec8eb..26ac5a5 100644 --- a/reference/infrastructure/networking/Controllers/UniFi Controller.md +++ b/deployments/Networking and Access/UniFi/Deploy UniFi Controller with Docker.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: The UniFi® Controller is a wireless network management software solution from Ubiquiti Networks™. It allows you to manage multiple wireless networks using a web browser. +## Purpose +The UniFi® Controller is a wireless network management software solution from Ubiquiti Networks™. It allows you to manage multiple wireless networks using a web browser. ```yaml title="docker-compose.yml" version: "2.1" @@ -46,3 +47,8 @@ networks: ```yaml title=".env" Not Applicable ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Ubuntu UniFi Installation Notes]() — Review the separate installation approach and its incomplete status. +- [Related Networking and Access Documentation](<../../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/infrastructure/networking/Controllers/UniFi Network Server Controller.md b/deployments/Networking and Access/UniFi/Deploy UniFi Network Server on Ubuntu.md similarity index 90% rename from reference/infrastructure/networking/Controllers/UniFi Network Server Controller.md rename to deployments/Networking and Access/UniFi/Deploy UniFi Network Server on Ubuntu.md index 8d2d9c7..da3a877 100644 --- a/reference/infrastructure/networking/Controllers/UniFi Network Server Controller.md +++ b/deployments/Networking and Access/UniFi/Deploy UniFi Network Server on Ubuntu.md @@ -4,7 +4,7 @@ tags: - Networking --- -**Purpose**: +## Purpose If you need to deploy Unifi Controller bare-metal into a virtual machine, you can do so with a few simple commands. You can feel free to reference the [original documentation](https://help.ui.com/hc/en-us/articles/220066768-Updating-and-Installing-Self-Hosted-UniFi-Network-Servers-Linux) if additional clarity is needed. !!! note "Assumptions" @@ -18,12 +18,12 @@ If you need to deploy Unifi Controller bare-metal into a virtual machine, you ca curl -sO https://get.glennr.nl/unifi/install/install_latest/unifi-latest.sh && bash unifi-latest.sh ``` - ## Install Components The installation will consist of a MongoDB server and a Unifi Network (controller) server. You will install the database first, then install the Unifi Controller second, so it can provision the newly-installed local MongoDB database server. ### General Configuration We need to configure APT with a few commands to ensure that we can download the MongoDB and Unifi packages. + ```sh sudo apt-get update && sudo apt-get install ca-certificates apt-transport-https echo 'deb [ arch=amd64,arm64 ] https://www.ui.com/downloads/unifi/debian stable ubiquiti' | sudo tee /etc/apt/sources.list.d/100-ubnt-unifi.list @@ -32,14 +32,19 @@ echo "deb [trusted=yes] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/3 sudo apt-get update ``` -!!! node "Alternative GPG Key Installation" +!!! note "Alternative GPG Key Installation" If you run into issues installing the GPG key for the Unifi packages, you can alternatively run the command seen below: + ```sh sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 06E85760C0A52C50 ``` ### MongoDB Server Run the following commands install and enable automatic startup for the MongoDB server. Original reference documentation can be found [here](https://www.mongodb.com/docs/manual/tutorial/install-mongodb-on-ubuntu/). + ```sh -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Networking and Access Documentation](<../../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/Networking and Access/index.md b/deployments/Networking and Access/index.md new file mode 100644 index 0000000..a7fefa6 --- /dev/null +++ b/deployments/Networking and Access/index.md @@ -0,0 +1,21 @@ +--- +tags: + - Networking and Access + - Deployments + - Documentation +--- + +# Networking and Access +## Purpose +Find deployments for networking and access. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- DNS +- NetBird +- Remote Access +- Reverse Proxies +- Sophos +- UniFi + +## Follow the Subject +[Networking and Access](<../../reference/Networking and Access/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/scripts/Bash/Install QEMU Guest Agent.md b/deployments/Virtualization and Storage/Guests/Install the QEMU Guest Agent.md similarity index 65% rename from scripts/Bash/Install QEMU Guest Agent.md rename to deployments/Virtualization and Storage/Guests/Install the QEMU Guest Agent.md index 2ecc3d6..ba5d65f 100644 --- a/scripts/Bash/Install QEMU Guest Agent.md +++ b/deployments/Virtualization and Storage/Guests/Install the QEMU Guest Agent.md @@ -6,14 +6,14 @@ tags: - Linux --- -**Purpose**: +## Purpose You may need to install the QEMU guest agent on linux VMs manually, while Windows-based devices work out-of-the-box after installing the VirtIO guest tools installer. === "Ubuntu Server" ```sh sudo su - apt update + apt update apt install -y qemu-guest-agent systemctl enable --now qemu-guest-agent ``` @@ -24,4 +24,7 @@ You may need to install the QEMU guest agent on linux VMs manually, while Window sudo su dnf install -y qemu-guest-agent systemctl enable --now qemu-guest-agent - ``` \ No newline at end of file + ``` + +## Related Documentation +- [Related Virtualization and Storage Documentation](<../../../reference/Virtualization and Storage/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/platforms/virtualization/Hyper V/Failover Cluster/Deploy Failover Cluster Node.md b/deployments/Virtualization and Storage/Hyper-V/Failover Cluster/Deploy Failover Cluster Node.md similarity index 69% rename from deployments/platforms/virtualization/Hyper V/Failover Cluster/Deploy Failover Cluster Node.md rename to deployments/Virtualization and Storage/Hyper-V/Failover Cluster/Deploy Failover Cluster Node.md index 0646962..514e6ac 100644 --- a/deployments/platforms/virtualization/Hyper V/Failover Cluster/Deploy Failover Cluster Node.md +++ b/deployments/Virtualization and Storage/Hyper-V/Failover Cluster/Deploy Failover Cluster Node.md @@ -1,15 +1,17 @@ --- tags: - - Documentation + - Virtualization and Storage + - Deploy Failover Cluster Node --- -**Purpose**: Deploying a Windows Server Node into the Hyper-V Failover Cluster is an essential part of rebuilding and expanding the backbone of my homelab. The documentation below goes over the process of setting up a bare-metal host from scratch and integrating it into the Hyper-V Failover Cluster. +## Purpose +Deploying a Windows Server Node into the Hyper-V Failover Cluster is an essential part of rebuilding and expanding the backbone of my homelab. The documentation below goes over the process of setting up a bare-metal host from scratch and integrating it into the Hyper-V Failover Cluster. !!! note "Prerequisites & Assumptions" This document assumes you are have installed and are running a bare-metal Hewlett-Packard Enterprise server with iLO (Integrated Lights Out) with the latest build of **Windows Server 2022 Datacenter (Desktop Experience)**. This document also assumes that you are adding an additional server node to an existing Hyper-V Failover Cluster. This document does not outline the exact process of setting up a Hyper-V Failover Cluster from-scratch, setting up a domain, DNS server, etc. Those are assumed to already exist in the environment. Your domain controller(s) need to be online and accessible from the Failover Cluster node you are building for things to work correctly. - + Download the newest build ISO of Windows Server 2022 at the [Microsoft Evaluation Center](https://go.microsoft.com/fwlink/p/?linkid=2195686&clcid=0x409&culture=en-us&country=us) ### Enable Remote Desktop @@ -19,14 +21,14 @@ Enable remote desktop however you can, but just be sure to disable NLA, see the This step can be disregarded if the domain controller(s) exist outside of the Hyper-V Failover Cluster. -``` powershell +```powershell # Enable Remote Desktop (NLA-Disabled) Set-ItemProperty -Path "HKLM:\System\CurrentControlSet\Control\Terminal Server" -Name "fDenyTSConnections" -Value 0 Set-ItemProperty -Path "HKLM:\System\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" -Name "UserAuthentication" -Value 0 Enable-NetFirewallRule -DisplayGroup "Remote Desktop" ``` ### Provision Server Roles, Activate, and Domain Join -``` powershell +```powershell # Rename the server Rename-Computer BUNNY-NODE-02 @@ -56,10 +58,10 @@ Restart-Computer You will need to start off by configuring a Switch Embedded Teaming (SET) team. This is the backbone that the server will use for all Guest VM traffic as well as remote-desktop access to the server node itself. You will need to rename the network adapters to make management easier. - Navigate to "Network Connections" then "Change Adapter Options" - * Rename the network adapters with simpler names. e.g. (`Ethernet 1` becomes `Port_1`) - * For the sake of demonstration, assume there are 2 10GbE NICs (`Port_1` and `Port_2`) + - Rename the network adapters with simpler names. e.g. (`Ethernet 1` becomes `Port_1`) + - For the sake of demonstration, assume there are 2 10GbE NICs (`Port_1` and `Port_2`) -``` powershell +```powershell # Create Switch Embedded Teaming (SET) team New-VMSwitch -Name Cluster_SET -NetAdapterName Port_1, Port_2 -EnableEmbeddedTeaming $true @@ -70,55 +72,60 @@ Get-NetAdapter | Where-Object { $_.Name -ne "vEthernet (Cluster_SET)" } | ForEac New-NetIPAddress -InterfaceAlias "vEthernet (Cluster_SET)" -IPAddress 192.168.3.5 -PrefixLength 24 -DefaultGateway 192.168.3.1 Set-DnsClientServerAddress -InterfaceAlias "vEthernet (Cluster_SET)" -ServerAddresses ("192.168.3.25","192.168.3.26") ``` + ### Configure iSCSI Initiator to Connect to TrueNAS Core Server At this point, now that we have verified that the 10GbE NICs can ping their respective iSCSI target server IP addresses, we can add them to the iSCSI Initiator in Server Manager which will allow us to mount the cluster storage for the Hyper-V Failover Cluster. - Open **Server Manager > MPIO** - * Navigate to the "Discover Multi-Paths" tab - * Check the "Add support for iSCSI devices" checkbox - * Click the "Add" button + - Navigate to the "Discover Multi-Paths" tab + - Check the "Add support for iSCSI devices" checkbox + - Click the "Add" button - Open **TrueNAS Core Server** - * Navigate to the [TrueNAS Core server](http://192.168.3.3) and add the "Initiator Name" seen on the "Configuration" tab of the iSCSI Initiator on the Virtualization Host to the `Sharing > iSCSI > Initiator Groups` > "iSCSI-Connected Servers" + - Navigate to the [TrueNAS Core server](http://192.168.3.3) and add the "Initiator Name" seen on the "Configuration" tab of the iSCSI Initiator on the Virtualization Host to the `Sharing > iSCSI > Initiator Groups` > "iSCSI-Connected Servers" - Open **iSCSI Initiator** - * Click on the "Discovery" tab - * Click the "Discover Portal" button - * Enter the IP addresses of "192.168.3.3". Leave the port as "3260". - * Example Initiator Name: `iqn.1991-05.com.microsoft:bunny-node-02.bunny-lab.io` - * Click the "Targets" tab to go back to the main page - * Click the "Refresh" button to display available iSCSI Targets - * Click on the first iSCSI Target `iqn.2005-10.org.moon-storage-01.ctl:iscsi-cluster-storage` then click the "Connect" button - * Check the "Enable Multi-Path" checkbox - * Click the "Advanced" button - * Click the "OK" button - * Navigate to "Disk Management" to bring the iSCSI drives "Online" (Dont do anything after this in Disk Management) + - Click on the "Discovery" tab + - Click the "Discover Portal" button + - Enter the IP addresses of "192.168.3.3". Leave the port as "3260". + - Example Initiator Name: `iqn.1991-05.com.microsoft:bunny-node-02.bunny-lab.io` + - Click the "Targets" tab to go back to the main page + - Click the "Refresh" button to display available iSCSI Targets + - Click on the first iSCSI Target `iqn.2005-10.org.moon-storage-01.ctl:iscsi-cluster-storage` then click the "Connect" button + - Check the "Enable Multi-Path" checkbox + - Click the "Advanced" button + - Click the "OK" button + - Navigate to "Disk Management" to bring the iSCSI drives "Online" (Dont do anything after this in Disk Management) ## Initialize and Join to Existing Failover-Cluster ### Validate Server is Ready to Join Cluster Now it is time to set up the Failover Cluster itself so we can join the server to the existing cluster. - Open **Server Manager** - * Click on the "Tools" dropdown menu - * Click on "Failover Cluster Manager" - * Click the "Validate Configuration" button in the middle of the window that appears - * Click "Next" - * Enter Server Name: `BUNNY-NODE-02.bunny-lab.io` - * Click the "Add" button, then "Next" - * Ensure "Run All Tests (Recommended)" is selected, then click "Next", then click "Next" to start. -### Join Server to Failover Cluster -* On the left-hand side, right-click on the "Failover Cluster Manager" in the tree - * Click on "Connect to Cluster" - * Enter `USAGI-CLUSTER.bunny-lab.io` - * Click "OK" -* Expand "USAGI-CLUSTER.bunny-lab.io" on the left-hand tree - * Right-click on "Nodes" - * Click "Add Node..." - * Click "Next" - * Enter Server Name: `BUNNY-NODE-02.bunny-lab.io` - * Click the "Add" button, then "Next" - * Ensure that "Run Configuration Validation Tests" radio box is checked, then click "Next" - * Validate that the node was successfully added to the Hyper-V Failover Cluster + - Click on the "Tools" dropdown menu + - Click on "Failover Cluster Manager" + - Click the "Validate Configuration" button in the middle of the window that appears + - Click "Next" + - Enter Server Name: `BUNNY-NODE-02.bunny-lab.io` + - Click the "Add" button, then "Next" + - Ensure "Run All Tests (Recommended)" is selected, then click "Next", then click "Next" to start. -## Cleanup & Final Touches -Ensure that you run all available Windows Updates before delegating guest VM roles to the new server in the failover cluster. This ensures you are up-to-date before you become reliant on the server for production operations. \ No newline at end of file +### Join Server to Failover Cluster +- On the left-hand side, right-click on the "Failover Cluster Manager" in the tree + - Click on "Connect to Cluster" + - Enter `USAGI-CLUSTER.bunny-lab.io` + - Click "OK" +- Expand "USAGI-CLUSTER.bunny-lab.io" on the left-hand tree + - Right-click on "Nodes" + - Click "Add Node..." + - Click "Next" + - Enter Server Name: `BUNNY-NODE-02.bunny-lab.io` + - Click the "Add" button, then "Next" + - Ensure that "Run Configuration Validation Tests" radio box is checked, then click "Next" + - Validate that the node was successfully added to the Hyper-V Failover Cluster + +## Cleanup and Final Touches +Ensure that you run all available Windows Updates before delegating guest VM roles to the new server in the failover cluster. This ensures you are up-to-date before you become reliant on the server for production operations. + +## Related Documentation +- [Related Virtualization and Storage Documentation](<../../../../reference/Virtualization and Storage/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/platforms/virtualization/OpenStack/Ansible OpenStack.md b/deployments/Virtualization and Storage/OpenStack/Ansible OpenStack.md similarity index 92% rename from deployments/platforms/virtualization/OpenStack/Ansible OpenStack.md rename to deployments/Virtualization and Storage/OpenStack/Ansible OpenStack.md index fef523b..d74424e 100644 --- a/deployments/platforms/virtualization/OpenStack/Ansible OpenStack.md +++ b/deployments/Virtualization and Storage/OpenStack/Ansible OpenStack.md @@ -4,11 +4,12 @@ tags: - Ansible --- +## Purpose +Deploying OpenStack via Ansible. + !!! warning "Document Under Construction" This document is very unfinished and should **NOT** be followed by anyone for deployment at this time. -**Purpose**: Deploying OpenStack via Ansible. - ## Required Hardware/Infrastructure Breakdown Every node in the OpenStack environment (including the deployment node) will be running Rocky Linux 9.5, as OpenStack Ansible only supports CentOS/RHEL/Rocky for its deployment. @@ -23,6 +24,7 @@ Every node in the OpenStack environment (including the deployment node) will be We want to ensure everything works even if the nodes have no internet access. By hardcoding the FQDNs, this protects us against several possible stupid situations. Run the following script to add the DNS entries. + ```sh # Make yourself root sudo su @@ -45,6 +47,7 @@ cat /etc/hosts !!! example "/etc/hosts Example Contents" When you run `cat /etc/hosts`, you should see output similar to the following: + ```ini title="/etc/hosts" 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 @@ -70,7 +73,7 @@ Ansible uses SSH with public key authentication to connect the deployment host a !!! warning "Do not run as root" You want to make sure you run these commands as a normal user. (e.g. `nicole`). -``` sh +```sh # Generate SSH Keys (Private / Public) ssh-keygen @@ -87,6 +90,7 @@ ssh nicole@openstack-node-03.bunny-lab.io ### Install the source and dependencies Install the source and dependencies for the deployment host. + ```sh sudo su git clone -b master https://opendev.org/openstack/openstack-ansible /opt/openstack-ansible @@ -95,7 +99,8 @@ bash scripts/bootstrap-ansible.sh ``` ### Disable Firewalld -The `firewalld` service is enabled on most CentOS systems by default and its default ruleset prevents OpenStack components from communicating properly. Stop the firewalld service and mask it to prevent it from starting. +The `firewalld` service is enabled on most CentOS systems by default and its default ruleset prevents OpenStack components from communicating properly. Stop the firewalld service and mask it to prevent it from starting. + ```sh systemctl stop firewalld systemctl mask firewalld @@ -106,12 +111,14 @@ Now we need to get the cluster/target nodes configured so that OpenStack can be ### Disable SELinux SELinux enabled is not currently supported in OpenStack-Ansible for CentOS/RHEL due to a lack of maintainers for the feature. + ```sh sudo sed -i 's/^SELINUX=enforcing/SELINUX=disabled/' /etc/sysconfig/selinux ``` ### Disable Firewalld -The `firewalld` service is enabled on most CentOS systems by default and its default ruleset prevents OpenStack components from communicating properly. Stop the firewalld service and mask it to prevent it from starting. +The `firewalld` service is enabled on most CentOS systems by default and its default ruleset prevents OpenStack components from communicating properly. Stop the firewalld service and mask it to prevent it from starting. + ```sh systemctl stop firewalld systemctl mask firewalld @@ -125,12 +132,14 @@ dnf install -y iputils lsof openssh-server sudo tcpdump python3 ### Reduce Kernel Logging Reduce the kernel log level by changing the printk value in your sysctls. + ```sh sudo echo "kernel.printk='4 1 7 4'" >> /etc/sysctl.conf ``` ### Configure Local Cinder/Ceph Storage (Optional if using iSCSI) At this point, we need to configure `/dev/sdb` as the local storage for Cinder. + ```sh pvcreate --metadatasize 2048 /dev/sdb vgcreate cinder-volumes /dev/sdb @@ -138,7 +147,8 @@ vgcreate cinder-volumes /dev/sdb !!! failure "`Cannot use /dev/sdb: device is partitioned`" You may (in rare cases) see the following error when trying to run `pvcreate --metadatasize 2048 /dev/sdb`, if that happens, just use `lsblk` to get the drive of the expected disk. In my example, we want the 500GB disk located at `/dev/sda`, seen in the example below: - ``` + + ```text [root@openstack-node-02 nicole]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 500G 0 disk @@ -153,4 +163,7 @@ vgcreate cinder-volumes /dev/sdb !!! question "End of Current Documentation" This is the end of where I have currently iterated in my lab and followed-along with the official documentation while generalizing it for my specific lab scenarios. The following link is where I am currently at/stuck and need to revisit at my earliest convenience. - https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/targethosts.html#configuring-the-network \ No newline at end of file + https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/targethosts.html#configuring-the-network + +## Related Documentation +- [Related Virtualization and Storage Documentation](<../../../reference/Virtualization and Storage/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/platforms/virtualization/OpenStack/Canonical OpenStack.md b/deployments/Virtualization and Storage/OpenStack/Canonical OpenStack.md similarity index 82% rename from deployments/platforms/virtualization/OpenStack/Canonical OpenStack.md rename to deployments/Virtualization and Storage/OpenStack/Canonical OpenStack.md index 44fc0f6..04bdb96 100644 --- a/deployments/platforms/virtualization/OpenStack/Canonical OpenStack.md +++ b/deployments/Virtualization and Storage/OpenStack/Canonical OpenStack.md @@ -1,9 +1,10 @@ --- tags: - OpenStack + - Virtualization and Storage --- -# OpenStack +## Purpose OpenStack is basically a virtual machine hypervisor that is HA and cluster-friendly. This particular variant is deployed via Canonical's MiniStack environment using SNAP. It will deploy OpenStack onto a single node, which can later be expanded to additional nodes. You can also use something like OpenShift to deploy a Kubernetes Cluster onto OpenStack automatically via its various APIs. **Reference Documentation**: @@ -13,33 +14,39 @@ OpenStack is basically a virtual machine hypervisor that is HA and cluster-frien !!! note This document assumes your bare-metal host server is running Ubuntu 22.04 LTS, has at least 16GB of Memory (**32GB for Multi-Node Deployments**), two network interfaces (one for management, one for remote VM access), 200GB of Disk Space for the root filesystem, another 200GB disk for Ceph distributed storage, and 4 processor cores. See [Single-Node Mode System Requirements](https://ubuntu.com/openstack/install) -!!! note Assumed Networking on the First Cluster Node +!!! note "Assumed Networking on the First Cluster Node" - **eth0** = 192.168.3.5 - **eth1** = 192.168.5.200 ### Update APT then install upgrades -``` +```sh sudo apt update && sudo apt upgrade -y && sudo apt install htop ncdu iptables nano -y ``` + !!! tip At this time, it would be a good idea to take a checkpoint/snapshot of the server (if it is a virtual machine). This gives you a starting point to come back to as you troubleshoot inevitable deployment issues. ### Update SNAP then install OpenStack SNAP -``` +```sh sudo snap refresh sudo snap install openstack --channel 2023.1 ``` -### Install & Configure Dependencies + +### Install and Configure Dependencies Sunbeam can generate a script to ensure that the machine has all of the required dependencies installed and is configured correctly for use in MicroStack. -``` + +```sh sunbeam prepare-node-script | bash -x && newgrp snap_daemon sudo reboot ``` + ### Bootstrapping Deploy the OpenStack cloud using the cluster bootstrap command. -``` + +```text sunbeam cluster bootstrap ``` + !!! warning If you get an "Unable to connect to websocket" error, run `sudo snap restart lxd`. [Known Bug Report](https://bugs.launchpad.net/snap-openstack/+bug/2033400) @@ -48,34 +55,39 @@ sunbeam cluster bootstrap Management networks shared by hosts = `192.168.3.0/24` MetalLB address allocation range (supports multiple ranges, comma separated) (10.20.21.10-10.20.21.20): `192.168.3.50-192.168.3.60` -### Cloud Initialization: +### Cloud Initialization - nicole@moon-stack-01:~$ `sunbeam configure --openrc demo-openrc` - Local or remote access to VMs [local/remote] (local): `remote` - CIDR of network to use for external networking (10.20.20.0/24): `192.168.5.0/24` - IP address of default gateway for external network (192.168.5.1): -- Populate OpenStack cloud with demo user, default images, flavors etc [y/n] (y): +- Populate OpenStack cloud with demo user, default images, flavors etc [y/n] (y): - Username to use for access to OpenStack (demo): `nicole` - Password to use for access to OpenStack (Vb********): `` -- Network range to use for project network (192.168.122.0/24): +- Network range to use for project network (192.168.122.0/24): - List of nameservers guests should use for DNS resolution (192.168.3.11 192.168.3.10): -- Enable ping and SSH access to instances? [y/n] (y): +- Enable ping and SSH access to instances? [y/n] (y): - Start of IP allocation range for external network (192.168.5.2): `192.168.5.201` - End of IP allocation range for external network (192.168.5.254): `192.168.5.251` -- Network type for access to external network [flat/vlan] (flat): +- Network type for access to external network [flat/vlan] (flat): - Free network interface that will be configured for external traffic: `eth1` -- WARNING: Interface eth1 is configured. Any configuration will be lost, are you sure you want to continue? [y/n]: y +- WARNING: Interface eth1 is configured. Any configuration will be lost, are you sure you want to continue? [y/n]: y ### Pull Down / Generate the Dashboard URL -``` +```text sunbeam openrc > admin-openrc sunbeam dashboard-url ``` -### Launch a Test VM: +### Launch a Test VM Verify the cloud by launching a VM called ‘test’ based on the ‘ubuntu’ image (Ubuntu 22.04 LTS). -``` + +```text sunbeam launch ubuntu --name test ``` -!!! note Sample output: + +!!! note "Sample output:" - Launching an OpenStack instance ... - Access instance with `ssh -i /home/ubuntu/.config/openstack/sunbeam ubuntu@10.20.20.200` + +## Related Documentation +- [Related Virtualization and Storage Documentation](<../../../reference/Virtualization and Storage/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/platforms/virtualization/proxmox/Configuring ISCSI Based Cluster Storage.md b/deployments/Virtualization and Storage/Proxmox/Configuring ISCSI Based Cluster Storage.md similarity index 92% rename from deployments/platforms/virtualization/proxmox/Configuring ISCSI Based Cluster Storage.md rename to deployments/Virtualization and Storage/Proxmox/Configuring ISCSI Based Cluster Storage.md index 73b82b2..0bf58f6 100644 --- a/deployments/platforms/virtualization/proxmox/Configuring ISCSI Based Cluster Storage.md +++ b/deployments/Virtualization and Storage/Proxmox/Configuring ISCSI Based Cluster Storage.md @@ -24,14 +24,13 @@ This approach is intended to: - All VM workloads are drained from at least one Proxmox node for maintenance !!! note "Proxmox VE Version Context" - This guide assumes **Proxmox VE 9.1.4 (or later)**. Snapshot-as-volume-chain support on shared LVM (e.g., iSCSI) is available and improved, including enhanced handling of vTPM state in offline snapshots. + This guide assumes **Proxmox VE 9.1.4 (or later)**. Snapshot-as-volume-chain support on shared LVM (e.g., iSCSI) is available and improved, including enhanced handling of vTPM state in offline snapshots. !!! warning "Important" - `volblocksize` **cannot be changed after zvol creation**. Choose carefully. + `volblocksize` **cannot be changed after zvol creation**. Choose carefully. ## Target Architecture - -``` +```text ZFS Pool └─ Zvol (Thick / Reserved) └─ iSCSI Extent @@ -41,7 +40,6 @@ ZFS Pool ``` ## Create a Dedicated Zvol for Proxmox - ### Variables Adjust as needed before execution. @@ -65,14 +63,12 @@ zfs create -V ${ZVOL_SIZE} \ The `refreservation` enforces **true thick provisioning** and prevents overcommit. ## Configure iSCSI Target (TrueNAS CORE) - This section uses a **hybrid approach**: - **CLI** is used for ZFS and LUN (extent backing) creation - **TrueNAS GUI** is used for iSCSI portal, target, and association - **CLI** is used again for validation ### Enable iSCSI Service - ```sh service ctld start sysrc ctld_enable=YES @@ -93,7 +89,6 @@ ctladm create -b block \ ``` ### Verify the LUN is real and correctly sized - ```sh ctladm devlist -v ``` @@ -102,7 +97,6 @@ ctladm devlist -v `Size (Blocks)` must be **non-zero** and match the zvol size. If it is `0`, stop and correct before proceeding. ### Configure iSCSI Portal, Target, and Extent Association (CLI Only) - !!! warning "Do NOT Use the TrueNAS iSCSI GUI" **Once you choose a CLI-managed iSCSI configuration, the TrueNAS Web UI must never be used for iSCSI.** Opening or modifying **Sharing → Block Shares (iSCSI)** in the GUI will **overwrite CTL runtime state**, invalidate manual `ctladm` configuration, and result in targets that appear correct but expose **no LUNs** to initiators. @@ -114,7 +108,6 @@ Opening or modifying **Sharing → Block Shares (iSCSI)** in the GUI will **over - Do **not** mix GUI extents with CLI-created LUNs #### Create iSCSI Portal (Listen on All Interfaces) - ```sh # Backup any existing ctl.conf cp -av /etc/ctl.conf /etc/ctl.conf.$(date +%Y%m%d-%H%M%S).bak 2>/dev/null || true @@ -185,9 +178,8 @@ lsblk ``` ## Create Shared LVM (Execute on One Node Only) - !!! warning "Important" - **Only run LVM creation on ONE node**. All other nodes will only scan. + **Only run LVM creation on ONE node**. All other nodes will only scan. ```sh # Initialize Physical Volume @@ -214,7 +206,6 @@ vgscan - Allow Snapshots as Volume-Chain: ✔️ ## Validation - - Snapshot create / revert / delete - Live migration between nodes - PBS backup and restore test @@ -226,7 +217,7 @@ vgscan If you need to expand the storage space of the newly-created iSCSI LUN, you can run the ZFS commands seen below on the TrueNAS Core server. The first command increases the size, the second command pre-allocated the space (thick-provisioned). !!! warning "ProxmoxVE Cluster-specific Notes" - + - `pvresize` must be executed on **exactly one** ProxmoxVE node. - All other nodes should only perform `pvscan` / `vgscan` after the resize. - Running `pvresize` on multiple nodes can corrupt shared LVM metadata. @@ -249,4 +240,7 @@ pvresize /dev/sdX # Rescan on the other nodes that you did not run the pvresize command on. They will now see the expanded free space. pvscan vgscan -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Proxmox Documentation](<../../../reference/Virtualization and Storage/Proxmox/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/platforms/virtualization/proxmox/Cloud Init Templates/Ubuntu Server.md b/deployments/Virtualization and Storage/Proxmox/Create an Ubuntu Cloud-Init Template.md similarity index 86% rename from deployments/platforms/virtualization/proxmox/Cloud Init Templates/Ubuntu Server.md rename to deployments/Virtualization and Storage/Proxmox/Create an Ubuntu Cloud-Init Template.md index 8cdcebb..86e83fd 100644 --- a/deployments/platforms/virtualization/proxmox/Cloud Init Templates/Ubuntu Server.md +++ b/deployments/Virtualization and Storage/Proxmox/Create an Ubuntu Cloud-Init Template.md @@ -7,8 +7,12 @@ tags: ## Purpose You may need to deploy many copies of a virtual machine rapidly, and don't want to go through the hassle of setting up everything ad-hoc as the needs arise for each VM workload. Creating a cloud-init template allows you to more rapidly deploy production-ready copies of a template VM (that you create below) into a ProxmoxVE environment. +!!! warning "Incomplete Template Procedure" + The package-configuration script and final hostname section are incomplete. Complete and validate those sections before relying on this page to produce a reusable template. + ### Download Image and Import into ProxmoxVE You will first need to pull down the OS image from Ubuntu's website via CLI, as there is currently no way to do this via the WebUI. Using SSH or the Shell within the WebUI of one of the ProxmoxVE servers, run the following commands to download and import the image into ProxmoxVE. + ```sh # Make a place to keep cloud images mkdir -p /var/lib/vz/template/images/ubuntu && cd /var/lib/vz/template/images/ubuntu @@ -36,8 +40,9 @@ qm set 9000 --scsi0 nfs-cluster-storage:9000/vm-9000-disk-0.qcow2 qm set 9000 --boot c --bootdisk scsi0 ``` -### Add Cloud-Init Drive & Configure Template Defaults -Now that the Ubuntu cloud image is attached as the VM’s primary disk, you need to attach a Cloud-Init drive. This special drive is where Proxmox writes your user data (username, SSH keys, network settings, etc.) at clone time. +### Add Cloud-Init Drive and Configure Template Defaults +Now that the Ubuntu cloud image is attached as the VM’s primary disk, you need to attach a Cloud-Init drive. This special drive is where Proxmox writes your user data (username, SSH keys, network settings, etc.) at clone time. + ```sh # Add a Cloud-Init drive to the VM qm set 9000 --ide2 nfs-cluster-storage:cloudinit @@ -69,7 +74,7 @@ qm set 9000 --sshkey /root/infrastructure_id_rsa.pub qm set 9000 --ipconfig0 ip=dhcp ``` -### Setup Packages in VM & Convert to Template +### Setup Packages in VM and Convert to Template At this point, we have a few things we need to do first before we can turn the VM into a template and make clones of it. You will need to boot up the VM we made (id 9000) and run the following commands to prepare it for becoming a template: ```sh @@ -100,10 +105,10 @@ You can now create new VMs instantly from the template we created above. - **Gateway (IPv4)**: `192.168.3.1` - Click the "**OK**" button - Start the VM and wait for it to automatically provision itself - + === "Via CLI" - ``` sh + ```sh # Create a new VM (example: VM 9100) cloned from the template qm clone 9000 9100 --name ubuntu-2404-test --full @@ -117,6 +122,10 @@ You can now create new VMs instantly from the template we created above. ### Configure VM Hostname At this point, the hostname of the VM will be randomized and you will probably want to set it to something statically, you can do that with the following commands after the server has finished starting: + ```sh -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Proxmox Documentation](<../../../reference/Virtualization and Storage/Proxmox/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/platforms/virtualization/proxmox/ProxmoxVE.md b/deployments/Virtualization and Storage/Proxmox/Deploy Proxmox VE.md similarity index 80% rename from deployments/platforms/virtualization/proxmox/ProxmoxVE.md rename to deployments/Virtualization and Storage/Proxmox/Deploy Proxmox VE.md index 768d51a..648fa9c 100644 --- a/deployments/platforms/virtualization/proxmox/ProxmoxVE.md +++ b/deployments/Virtualization and Storage/Proxmox/Deploy Proxmox VE.md @@ -1,10 +1,17 @@ --- tags: - Proxmox + - Virtualization and Storage --- +## Purpose +This document records the procedure for deploy proxmox ve. Follow the environment assumptions and commands below. + +!!! info "Recorded Host Environment" + This deployment records the earlier Proxmox VE 8.1 and `MOONGATE.local` environment. Its backplane addresses differ from the separate shared-storage guide. Confirm the target host version and network plan before combining these instructions. + ## Initial Installation / Configuration -Proxmox Virtual Environment is an open source server virtualization management solution based on QEMU/KVM and LXC. You can manage virtual machines, containers, highly available clusters, storage and networks with an integrated, easy-to-use web interface or via CLI. +Proxmox Virtual Environment is an open source server virtualization management solution based on QEMU/KVM and LXC. You can manage virtual machines, containers, highly available clusters, storage and networks with an integrated, easy-to-use web interface or via CLI. !!! note This document assumes you have a storage server that hosts both ISO files via CIFS/SMB share, and has the ability to set up an iSCSI LUN (VM & Container storage). This document assumes that you are using a TrueNAS Core server to host both of these services. @@ -14,13 +21,14 @@ You will need to download the [Proxmox VE 8.1 ISO Installer](https://www.proxmox !!! warning If you are virtualizing Proxmox under a Hyper-V environment, you will need to follow the [Official Documentation](https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/user-guide/enable-nested-virtualization) to ensure that nested virtualization is enabled. An example is listed below: - ``` + + ```powershell Set-VMProcessor -VMName -ExposeVirtualizationExtensions $true # (1) Get-VMNetworkAdapter -VMName | Set-VMNetworkAdapter -MacAddressSpoofing On # (2) - ``` + ``` 1. This tells Hyper-V to allow the GuestVM to behave as a hypervisor, nested under Hyper-V, allowing the virtualization functionality of the Hypervisor's CPU to be passed-through to the GuestVM. - 2. This tells Hyper-V to allow your GuestVM to have multiple nested virtual machines with their own independant MAC addresses. This is useful when using nested Virtual Machines, but is also a requirement when you set up a [Docker Network](../../../../reference/infrastructure/networking/docker-networking/docker-networking.md) leveraging MACVLAN technology. + 2. This tells Hyper-V to allow your GuestVM to have multiple nested virtual machines with their own independant MAC addresses. This is useful when using nested Virtual Machines, but is also a requirement when you set up a [Docker Network](<../../Containers/Docker/Create the Docker Network.md>) leveraging MACVLAN technology. ### Networking You will need to set a static IP address, in this case, it will be an address within the 20GbE network. You will be prompted to enter these during the ProxmoxVE installation. Be sure to set the hostname to something that matches the following FQDN: `proxmox-node-01.MOONGATE.local`. @@ -35,7 +43,7 @@ You will need to set a static IP address, in this case, it will be an address wi ### iSCSI Initator Configuration You will need to add the iSCSI initiator from the proxmox node to the allowed initiator list in TrueNAS Core under "**Sharing > Block Shares (iSCSI) > Initiators Groups**" -In this instance, we will reference Group ID: `2`. We need to add the iniator to the "**Allowed Initiators (IQN)**" section. This also includes the following networks that are allowed to connect to the iSCSI portal: +In this instance, we will reference Group ID: `2`. We need to add the iniator to the "**Allowed Initiators (IQN)**" section. This also includes the following networks that are allowed to connect to the iSCSI portal: - `192.168.101.0/24` - `192.168.102.0/24` @@ -47,7 +55,8 @@ To get the iSCSI Initiator IQN of the current Proxmox node, you need to navigate - On the left-hand side, click on the name of the server node (e.g. `proxmox-node-01` or `proxmox-node-02`) - Click on "**Shell**" to open a CLI to the server - Run the following command to get the iSCSI Initiator (IQN) name to give to TrueNAS Core for the previously-mentioned steps: -``` sh + +```sh cat /etc/iscsi/initiatorname.iscsi | grep "InitiatorName=" | sed 's/InitiatorName=//' ``` @@ -58,7 +67,8 @@ cat /etc/iscsi/initiatorname.iscsi | grep "InitiatorName=" | sed 's/InitiatorNam You will likely not be paying for / using the enterprise subscription, so we are going to disable that functionality and enable unstable builds. The unstable builds are surprisingly stable, and should not cause you any issues. Add Unstable Update Repository: -```jsx title="/etc/apt/sources.list" + +```text title="/etc/apt/sources.list" # Add to the end of the file # Non-Production / Unstable Updates deb https://download.proxmox.com/debian bookworm pve-no-subscription @@ -68,29 +78,33 @@ deb https://download.proxmox.com/debian bookworm pve-no-subscription Please note the reference to `bookworm` in both the sections above and below this notice, this may be different depending on the version of ProxmoxVE you are deploying. Please reference the version indicated by the rest of the entries in the sources.list file to know which one to use in the added line section. Comment-Out Enterprise Repository: -```jsx title="/etc/apt/sources.list.d/pve-enterprise.list" + +```text title="/etc/apt/sources.list.d/pve-enterprise.list" # deb https://enterprise.proxmox.com/debian/pve bookworm pve-enterprise ``` Pull / Install Available Updates: -``` sh + +```sh apt-get update apt dist-upgrade reboot ``` ## NIC Teaming -You will need to set up NIC teaming to configure a LACP LAGG. This will add redundancy and a way for devices outside of the 20GbE backplane to interact with the server. +You will need to set up NIC teaming to configure a LACP LAGG. This will add redundancy and a way for devices outside of the 20GbE backplane to interact with the server. - Ensure that all of the network interfaces appear as something similar to the following: -```jsx title="/etc/network/interfaces" + +```text title="/etc/network/interfaces" iface eno1 inet manual iface eno2 inet manual # etc ``` - Adjust the network interfaces to add a bond: -```jsx title="/etc/network/interfaces" + +```text title="/etc/network/interfaces" auto eno1 iface eno1 inet manual @@ -120,33 +134,33 @@ iface vmbr0 inet static - Reboot the server again to make the networking changes take effect fully. Use iLO / iDRAC / IPMI if you have that functionality on your server in case your configuration goes errant and needs manual intervention / troubleshooting to re-gain SSH control of the proxmox server. -## Generalizing VMs for Cloning / Templating: +## Generalizing VMs for Cloning / Templating These are the commands I run after cloning a Linux machine so that it resets all information for the machine it was cloned from. !!! note If you use cloud-init-aware OS images as described under Cloud-Init Support on https://pve.proxmox.com/pve-docs/chapter-qm.html, these steps won’t be necessary! -```jsx title="Change Hostname" +```sh title="Change Hostname" sudo nano /etc/hostname ``` -```jsx title="Change Hosts File" +```sh title="Change Hosts File" sudo nano /etc/hosts ``` -```jsx title="Reset the Machine ID" +```text title="Reset the Machine ID" rm -f /etc/machine-id /var/lib/dbus/machine-id dbus-uuidgen --ensure=/etc/machine-id dbus-uuidgen --ensure ``` -```jsx title="Regenerate SSH Keys" +```text title="Regenerate SSH Keys" rm -f /etc/machine-id /var/lib/dbus/machine-id dbus-uuidgen --ensure=/etc/machine-id dbus-uuidgen --ensure ``` -```jsx title="Reboot the Server to Apply Changes" +```text title="Reboot the Server to Apply Changes" reboot ``` @@ -155,3 +169,5 @@ Setting up alerts in Proxmox is important and critical to making sure you are no https://technotim.live/posts/proxmox-alerts/ +## Related Documentation +- [Related Proxmox Documentation](<../../../reference/Virtualization and Storage/Proxmox/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/platforms/virtualization/proxmox/ZFS over ISCSI.md b/deployments/Virtualization and Storage/Proxmox/ZFS over ISCSI.md similarity index 89% rename from deployments/platforms/virtualization/proxmox/ZFS over ISCSI.md rename to deployments/Virtualization and Storage/Proxmox/ZFS over ISCSI.md index c5262f9..ce1e253 100644 --- a/deployments/platforms/virtualization/proxmox/ZFS over ISCSI.md +++ b/deployments/Virtualization and Storage/Proxmox/ZFS over ISCSI.md @@ -5,7 +5,8 @@ tags: - iSCSI --- -**Purpose**: There is a way to incorporate ProxmoxVE and TrueNAS more deeply using SSH, simplifying the deployment of virtual disks/volumes passed into GuestVMs in ProxmoxVE. Using ZFS over iSCSI will give you the following non-exhaustive list of benefits: +## Purpose +There is a way to incorporate ProxmoxVE and TrueNAS more deeply using SSH, simplifying the deployment of virtual disks/volumes passed into GuestVMs in ProxmoxVE. Using ZFS over iSCSI will give you the following non-exhaustive list of benefits: - Automatically make Zvols in a ZFS Storage Pool - Automatically bind device-based iSCSI Extents/LUNs to the Zvols @@ -20,7 +21,7 @@ tags: Secondly, this guide assumes the ProxmoxVE cluster nodes and TrueNAS server exist on the same network `192.168.101.0/24`. ## ZFS over iSCSI Operational Flow -``` mermaid +```mermaid sequenceDiagram participant ProxmoxVE as ProxmoxVE Cluster participant TrueNAS as TrueNAS Core (inc. iSCSI & ZFS Storage) @@ -39,16 +40,16 @@ You first need to make some changes to the SSHD configuration of the ZFS server( === "OpenSSH-based OS" - ```jsx title="/etc/ssh/sshd_config" + ```text title="/etc/ssh/sshd_config" UseDNS no GSSAPIAuthentication no ``` - + === "Solaris-based OS" - ```jsx title="/etc/ssh/sshd_config" - LookupClientHostnames no - VerifyReverseMapping no + ```text title="/etc/ssh/sshd_config" + LookupClientHostnames no + VerifyReverseMapping no GSSAPIAuthentication no ``` @@ -56,23 +57,23 @@ You first need to make some changes to the SSHD configuration of the ZFS server( ### Configure SSH Key Exchange The first step is creating SSH trust between the ProxmoxVE cluster nodes and the TrueNAS storage appliance. You will leverage the ProxmoxVE `shell` on every node of the cluster to run the following commands. -**Note**: I will be writing the SSH configuration with the name `192.168.101.100` for simplicity so I know what server the identity belongs to. You could also name it something else like `storage.bunny-lab.io_id_rsa`. +**Note**: I will be writing the SSH configuration with the name `192.168.101.100` for simplicity so I know what server the identity belongs to. You could also name it something else like `storage.bunny-lab.io_id_rsa`. -``` sh +```sh mkdir /etc/pve/priv/zfs ssh-keygen -f /etc/pve/priv/zfs/192.168.101.100_id_rsa # (1) ssh-copy-id -i /etc/pve/priv/zfs/192.168.101.100_id_rsa.pub root@192.168.101.100 # (2) ssh -i /etc/pve/priv/zfs/192.168.101.100_id_rsa root@192.168.101.100 # (3) ``` - + 1. Do not set a password. It will break the automatic functionality. 2. Send the SSH key to the TrueNAS server. 3. Connect to the TrueNAS server at least once to finish establishing the connection. -### Install & Configure Storage Provider +### Install and Configure Storage Provider Now you need to set up the storage provider in TrueNAS. You will run the commands below within a ProxmoxVE shell, then when finished, log out of the ProxmoxVE WebUI, clear the browser cache for ProxmoxVE, then log back in. This will have added a new storage provider called `FreeNAS-API` under the `ZFS over iSCSI` storage type. -``` sh +```sh keyring_location=/usr/share/keyrings/ksatechnologies-truenas-proxmox-keyring.gpg curl -1sLf 'https://dl.cloudsmith.io/public/ksatechnologies/truenas-proxmox/gpg.284C106104A8CE6D.key' | gpg --dearmor >> ${keyring_location} @@ -120,4 +121,7 @@ Navigate to **"Datacenter (BUNNY-CLUSTER) > Storage > Add > ZFS over iSCSI"** | Nodes | `proxmox-node-01,proxmox-node-02` | All ProxmoxVE Cluster Nodes | !!! success "Storage is Provisioned" - At this point, the storage should propagate throughout the ProxmoxVE cluster, and appear as a location to deploy virtual machines and/or containers. You can now use this storage for snapshots and live-migrations between ProxmoxVE cluster nodes as well. \ No newline at end of file + At this point, the storage should propagate throughout the ProxmoxVE cluster, and appear as a location to deploy virtual machines and/or containers. You can now use this storage for snapshots and live-migrations between ProxmoxVE cluster nodes as well. + +## Related Documentation +- [Related Proxmox Documentation](<../../../reference/Virtualization and Storage/Proxmox/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/Virtualization and Storage/Rancher Harvester/Harvester.md b/deployments/Virtualization and Storage/Rancher Harvester/Harvester.md new file mode 100644 index 0000000..9619166 --- /dev/null +++ b/deployments/Virtualization and Storage/Rancher Harvester/Harvester.md @@ -0,0 +1,64 @@ +--- +tags: + - Rancher + - Harvester +--- + +## Purpose +Rancher Harvester is an awesome tool that acts like a self-hosted cloud VDI provider, similar to AWS, Linode, and other online cloud compute platforms. In most scenarios, you will deploy "Rancher" in addition to Harvester to orchestrate the deployment, management, and rolling upgrades of a Kubernetes Cluster. You can also just run standalone Virtual Machines, similar to Hyper-V, RHEV, oVirt, Bhyve, XenServer, XCP-NG, and VMware ESXi. + +!!! note "Prerequisites" + This document assumes your bare-metal host has at least 32GB of Memory, 200GB of Disk Space, and 8 processor cores. See [Recommended System Requirements](https://docs.harvesterhci.io/v1.1/install/requirements) + +## First Harvester Node +### Download Installer ISO +You will need to navigate to the Rancher Harvester GitHub to download the [latest ISO release of Harvester](https://releases.rancher.com/harvester/v1.1.2/harvester-v1.1.2-amd64.iso), currently **v1.1.2**. Then image it onto a USB flashdrive using a tool like [Rufus](https://github.com/pbatard/rufus/releases/download/v4.2/rufus-4.2p.exe). Proceed to boot the bare-metal server from the USB drive to begin the Harvester installation process. + +### Begin Setup Process +You will be waiting a few minutes while the server boots from the USB drive, but you will eventually land on a page where it asks you to set up various values to use for networking and the cluster itself. +The values seen below are examples and represent how my homelab is configured. +- **Management Interface(s)**: `eno1,eno2,eno3,eno4` +- **Network Bond Mode**: `Active-Backup` +- **IP Address**: `192.168.3.254/24` *<---- **Note:** Be sure to add CIDR Notation*. +- **Gateway**: `192.168.3.1` +- **DNS Server(s)**: `1.1.1.1,1.0.0.1,8.8.8.8,8.8.4.4` +- **Cluster VIP (Virtual IP)**: `192.168.3.251` *<---- **Note**: See "VIRTUAL IP CONFIGURATION" note below.* +- **Cluster Node Token**: `19-USED-when-JOINING-more-NODES-to-EXISTING-cluster-55` +- **NTP Server(s)**: `0.suse.pool.ntp.org` + +!!! warning "Virtual IP Configuration" + The VIP assigned to the first node in the cluster will act as a proxy to the built-in load-balancing system. It is important that you do not create a second node with the same VIP (Could cause instability in existing cluster), or use an existing VIP as the Node IP address of a new Harvester Cluster Node. + +!!! tip "Tip" + Based on your preference, it would be good to assign the device a static DHCP reservation, or use numbers counting down from **.254** (e.g. `192.168.3.254`, `192.168.3.253`, `192.168.3.252`, etc...) + +### Wait for Installation to Complete +The installation process will take quite some time, but when it is finished, the Harvester Node will reboot and take you to a splash screen with the Harvester logo, with indicators as to what the VIP and Management Interface IPs are configured as, and whether or not the associated systems are operational and ready. **Be patient until both statuses say `READY`**. If after 15 minutes the status has still not changed to `READY` both for fields, see the note below. +!!! warning "Issues with `rancher-harvester-repo` Image" + During my initial deployment efforts with Harvester v.1.1.2, I noticed that the Harvester Node never came online. That was because something bugged-out during installation and the `rancher-harvester-repo` image was not properly installed prior to node initialization. This will effectively soft-lock the node unless you reinstall the node from scratch, as the Docker Hub Registry that Harvester is looking for to finish the deployment does not exist anymore and depends on the local image bundled with the installer ISO. + + If this happens, you unfortunately need to start over and reinstall Harvester and hope that it works the second time around. No other workarounds are currently known at this time on version 1.1.2. + +## Additional Harvester Nodes +If you work in a production environment, you will want more than one Harvester node to allow live-migrations, high-availability, and better load-balancing in the Harvester Cluster. The section below will outline the steps necessary to create additional Harvester nodes, join them to the existing Harvester cluster, and validate that they are functioning without issues. + +### Installation Process +Not Documented Yet + +### Joining Node to Existing Cluster +Not Documented Yet + +## Installing Rancher +If you plan on using Harvester for more than just running Virtual Machines (e.g. Containers), you will want to deploy Rancher inside of the Harvester Cluster in order or orchestrate the deployment, management, and rolling upgrades of various forms of Kubernetes Clusters (RKE2 Suggested). The steps below will go over the process of deploying a High-Availability Rancher environment to "adopt" Harvester as a VDI/compute platform for deploying the Kubernetes Cluster. + +### Provision ControlPlane Node(s) VMs on Harvester +Not Documented Yet + +### Adopt Harvester as Cluster Target +Not Documented Yet + +### Deploy Production Kubernetes Cluster to Harvester +Not Documented Yet + +## Related Documentation +- [Related Virtualization and Storage Documentation](<../../../reference/Virtualization and Storage/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/Virtualization and Storage/index.md b/deployments/Virtualization and Storage/index.md new file mode 100644 index 0000000..fe06451 --- /dev/null +++ b/deployments/Virtualization and Storage/index.md @@ -0,0 +1,20 @@ +--- +tags: + - Virtualization and Storage + - Deployments + - Documentation +--- + +# Virtualization and Storage +## Purpose +Find deployments for virtualization and storage. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- Guests +- Hyper-V +- OpenStack +- Proxmox +- Rancher Harvester + +## Follow the Subject +[Virtualization and Storage](<../../reference/Virtualization and Storage/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/workflows/operations/Linux/Fedora Workstation/Full Setup.md b/deployments/Windows and Linux/Fedora/Set Up the Fedora Workstation.md similarity index 90% rename from workflows/operations/Linux/Fedora Workstation/Full Setup.md rename to deployments/Windows and Linux/Fedora/Set Up the Fedora Workstation.md index c867c6c..74d6d23 100644 --- a/workflows/operations/Linux/Fedora Workstation/Full Setup.md +++ b/deployments/Windows and Linux/Fedora/Set Up the Fedora Workstation.md @@ -5,11 +5,11 @@ tags: - Workstation --- -**Purpose**: +## Purpose This document serves as a general guideline for my workstation deployment process when working with Fedora Workstation 41 and up. This document will constantly evolve over time based on my needs. ## Automate Initial Configurations -```sh +```sh # Set Hostname sudo hostnamectl set-hostname lab-desktop-01 @@ -35,7 +35,7 @@ sudo reboot The problem with Fast Boot is that it effectively leaves the shared disks between Windows and Linux in a locked read-only state, which makes installing Steam games and software impossible. ## Manually Address Remaining Things -At this point, we need to do some manual work, since not everything can be handled by the terminal. +At this point, we need to do some manual work, since not everything can be handled by the terminal. ### Install Software (Software Manager) Now we need to install a few things: @@ -55,4 +55,7 @@ Backups are located here: https://cloud.bunny-lab.io/f/792649 ### Configure Window Snapping By default, XFCE has a really small threshold for telling windows to "snap" to the sides of the screens, such as a half:half arrangement. This can be adjusted by navigating to "**Applications Menu > Settings > Settings Manager > Windows Manager Tweaks > Placement**" -Once you have reached this window, you will see a slider from "**Small**" to "**Large**". Slide the slider all the way to the right, facing "**Large**". Now windows will snap to the sides of the screen successfully. \ No newline at end of file +Once you have reached this window, you will see a slider from "**Small**" to "**Large**". Slide the slider all the way to the right, facing "**Large**". Now windows will snap to the sides of the screen successfully. + +## Related Documentation +- [Related Windows and Linux Documentation](<../../../reference/Windows and Linux/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/Windows and Linux/index.md b/deployments/Windows and Linux/index.md new file mode 100644 index 0000000..77cbe79 --- /dev/null +++ b/deployments/Windows and Linux/index.md @@ -0,0 +1,16 @@ +--- +tags: + - Windows and Linux + - Deployments + - Documentation +--- + +# Windows and Linux +## Purpose +Find deployments for windows and linux. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- Fedora + +## Follow the Subject +[Windows and Linux](<../../reference/Windows and Linux/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/deployments/automation/ansible/awx/Deployment/AWX Operator.md b/deployments/automation/AWX/AWX Operator.md similarity index 85% rename from deployments/automation/ansible/awx/Deployment/AWX Operator.md rename to deployments/automation/AWX/AWX Operator.md index 64f16e3..dcbf474 100644 --- a/deployments/automation/ansible/awx/Deployment/AWX Operator.md +++ b/deployments/automation/AWX/AWX Operator.md @@ -5,14 +5,14 @@ tags: - Automation --- -**Purpose**: +## Purpose Deploying a Rancher RKE2 Cluster-based Ansible AWX Operator server. This can scale to a larger more enterprise environment if needed. -!!! note Prerequisites +!!! note "Prerequisites" This document assumes you are running **Ubuntu Server 22.04** or later with at least 16GB of memory, 8 CPU cores, and 64GB of storage. ## Deploy Rancher RKE2 Cluster -You will need to deploy a [Rancher RKE2 Cluster](../../../../platforms/containerization/kubernetes/deployment/rancher-rke2.md) on an Ubuntu Server-based virtual machine. After this phase, you can focus on the Ansible AWX-specific deployment. A single ControlPlane node is all you need to set up AWX, additional infrastructure can be added after-the-fact. +You will need to deploy a [Rancher RKE2 Cluster](<../../Containers/Kubernetes/Rancher RKE2.md>) on an Ubuntu Server-based virtual machine. After this phase, you can focus on the Ansible AWX-specific deployment. A single ControlPlane node is all you need to set up AWX, additional infrastructure can be added after-the-fact. !!! tip "Checkpoint/Snapshot Reminder" If this is a virtual machine, after deploying the RKE2 cluster and validating it functions, now would be the best time to take a checkpoint / snapshot of the VM before moving forward, in case you need to perform rollbacks of the server(s) if you accidentally misconfigure something during deployment. @@ -20,7 +20,7 @@ You will need to deploy a [Rancher RKE2 Cluster](../../../../platforms/container ## Server Configuration The AWX deployment will consist of 3 yaml files that configure the containers for AWX as well as the NGINX ingress networking-side of things. You will need all of them in the same folder for the deployment to be successful. For the purpose of this example, we will put all of them into a folder located at `/awx`. -``` sh +```sh # Make the deployment folder mkdir -p /awx cd /awx @@ -28,20 +28,23 @@ cd /awx We need to increase filesystem access limits: Temporarily Set the Limits Now: -``` sh + +```sh sudo sysctl fs.inotify.max_user_watches=524288 sudo sysctl fs.inotify.max_user_instances=512 ``` Permanently Set the Limits for Later: -```jsx title="/etc/sysctl.conf" + +```ini title="/etc/sysctl.conf" # fs.inotify.max_user_watches = 524288 fs.inotify.max_user_instances = 512 ``` Apply the Settings: -``` sh + +```sh sudo sysctl -p ``` @@ -96,32 +99,36 @@ You will need to create these files all in the same directory using the content ``` ## Ensure the Kubernetes Cluster is Ready -Check that the status of the cluster is ready by running the following commands, it should appear similar to the [Rancher RKE2 Example](../../../../platforms/containerization/kubernetes/deployment/rancher-rke2.md#install-helm-rancher-certmanager-jetstack-rancher-and-longhorn): -``` +Check that the status of the cluster is ready by running the following commands, it should appear similar to the [Rancher RKE2 Example](<../../Containers/Kubernetes/Rancher RKE2.md#install-helm-rancher-certmanager-jetstack-rancher-and-longhorn>): + +```sh export KUBECONFIG=/etc/rancher/rke2/rke2.yaml kubectl get pods --all-namespaces ``` ## Ensure the Timezone / Date is Accurate You want to make sure that the Kubernetes environment and Node itself have accurate time for a number of reasons, least of which, is if you are using Ansible with Kubernetes authentication, if the date/time is inaccurate, things will not work correctly. -``` sh + +```sh sudo timedatectl set-timezone America/Denver ``` ## Deploy AWX using Kustomize -Now it is time to tell Kubernetes to read the configuration files using Kustomize (*built-in to newer versions of Kubernetes*) to deploy AWX into the cluster. +Now it is time to tell Kubernetes to read the configuration files using Kustomize (*built-in to newer versions of Kubernetes*) to deploy AWX into the cluster. !!! warning "Be Patient" - The AWX deployment process can take a while. Use the commands in the [Troubleshooting](./awx-operator.md#troubleshooting) section if you want to track the progress after running the commands below. + The AWX deployment process can take a while. Use the commands in the [Troubleshooting]() section if you want to track the progress after running the commands below. If you get an error that looks like the below, re-run the `kubectl apply -k .` command a second time after waiting about 10 seconds. The second time the error should be gone. - ``` sh + + ```sh error: resource mapping not found for name: "awx" namespace: "awx" from ".": no matches for kind "AWX" in version "awx.ansible.com/v1beta1" ensure CRDs are installed first ``` To check on the progress of the deployment, you can run the following command: `kubectl get pods -n awx` You will know that AWX is ready to be accessed in the next step if the output looks like below: - ``` + + ```text NAME READY STATUS RESTARTS AGE awx-operator-controller-manager-7b9ccf9d4d-cnwhc 2/2 Running 2 (3m41s ago) 9m41s awx-postgres-13-0 1/1 Running 0 6m12s @@ -129,18 +136,18 @@ Now it is time to tell Kubernetes to read the configuration files using Kustomiz awx-web-6dbd7df9f7-kn8k2 3/3 Running 0 93s ``` -``` sh +```sh cd /awx kubectl apply -k . ``` !!! warning "Be Patient - Wait 20 Minutes" - The process may take a while to spin up AWX, postgresql, redis, and other workloads necessary for AWX to function. Depending on the speed of the server, it may take between 5 and 20 minutes for AWX to be ready to connect to. You can watch the progress via the CLI commands listed above, or directly on Rancher's WebUI at https://rancher.bunny-lab.io. + The process may take a while to spin up AWX, postgresql, redis, and other workloads necessary for AWX to function. Depending on the speed of the server, it may take between 5 and 20 minutes for AWX to be ready to connect to. You can watch the progress via the CLI commands listed above, or directly on Rancher's WebUI at https://rancher.bunny-lab.io. ## Access the AWX WebUI behind Ingress Controller -After you have deployed AWX into the cluster, it will not be immediately accessible to the host's network (such as your personal computer) unless you set up a DNS record pointing to it. In the example above, you would have an `A` or `CNAME` DNS record pointing to the internal IP address of the Rancher RKE2 Cluster host. +After you have deployed AWX into the cluster, it will not be immediately accessible to the host's network (such as your personal computer) unless you set up a DNS record pointing to it. In the example above, you would have an `A` or `CNAME` DNS record pointing to the internal IP address of the Rancher RKE2 Cluster host. -The RKE2 Cluster will translate `awx.bunny-lab.io` to the AWX web-service container(s) automatically due to having an internal Reverse Proxy within the Kubernetes Cluster. SSL certificates generated within Kubernetes/Rancher RKE2 are not covered in this documentation, but suffice to say, the AWX server can be configured on behind another reverse proxy such as Traefik or via Cert-Manager / JetStack. The process of setting this up goes outside the scope of this document. +The RKE2 Cluster will translate `awx.bunny-lab.io` to the AWX web-service container(s) automatically due to having an internal Reverse Proxy within the Kubernetes Cluster. SSL certificates generated within Kubernetes/Rancher RKE2 are not covered in this documentation, but suffice to say, the AWX server can be configured on behind another reverse proxy such as Traefik or via Cert-Manager / JetStack. The process of setting this up goes outside the scope of this document. ### Traefik Implementation If you want to put this behind traefik, you will need a slightly unique traefik configuration file, seen below, to effectively transparently passthrough traffic into the RKE2 Cluster's reverse proxy. @@ -168,16 +175,18 @@ tcp: If you have gotten this far, you should now be able to access AWX via the WebUI and log in. - AWX WebUI: https://awx.bunny-lab.io - ![Ansible AWX WebUI](../awx.png) + ![Ansible AWX WebUI]() You may see a prompt about "AWX is currently upgrading. This page will refresh when complete". Be patient, let it finish. When it's done, it will take you to a login page. - AWX will generate its own secure password the first time you set up AWX. Username is `admin`. You can run the following command to retrieve the password: - ``` + AWX will generate its own secure password the first time you set up AWX. Username is `admin`. You can run the following command to retrieve the password: + + ```text kubectl get secret awx-admin-password -n awx -o jsonpath="{.data.password}" | base64 --decode ; echo ``` ## Change Admin Password You will want to change the admin password straight-away. Use the following navigation structure to find where to change the password: -``` mermaid + +```mermaid graph LR A[AWX Dashboard] --> B[Access] B --> C[Users] @@ -190,9 +199,14 @@ You may wish to want to track the deployment process to verify that it is actual ### AWX-Manager Deployment Logs You may want to track the internal logs of the `awx-manager` container which is responsible for the majority of the automated deployment of AWX. You can do so by running the command below. -``` + +```text kubectl logs -n awx awx-operator-controller-manager-6c58d59d97-qj2n2 -c awx-manager ``` + !!! note The `-6c58d59d97-qj2n2` noted at the end of the Kubernetes "Pod" mentioned in the command above is randomized. You will need to change it based on the name shown when running the `kubectl get pods -n awx` command. +## Related Documentation +- [Related AWX Documentation](<../../../reference/Automation/AWX/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/automation/ansible/awx/Deployment/AWX in Minikube.md b/deployments/automation/AWX/AWX in Minikube.md similarity index 66% rename from deployments/automation/ansible/awx/Deployment/AWX in Minikube.md rename to deployments/automation/AWX/AWX in Minikube.md index fcb5bcc..adf0854 100644 --- a/deployments/automation/ansible/awx/Deployment/AWX in Minikube.md +++ b/deployments/automation/AWX/AWX in Minikube.md @@ -5,14 +5,14 @@ tags: - Automation --- -# Deploy AWX on Minikube Cluster +## Purpose Minikube Cluster based deployment of Ansible AWX. (Ansible Tower) -!!! note Prerequisites +!!! note "Prerequisites" This document assumes you are running **Ubuntu Server 20.04** or later. ## Install Minikube Cluster ### Update the Ubuntu Server -``` +```sh sudo apt update sudo apt upgrade -y sudo apt autoremove -y @@ -20,7 +20,8 @@ sudo apt autoremove -y ### Download and Install Minikube (Ubuntu Server) Additional Documentation: https://minikube.sigs.k8s.io/docs/start/ -``` + +```sh curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb sudo dpkg -i minikube_latest_amd64.deb @@ -30,28 +31,31 @@ sudo apt install docker.io nfs-common iptables nano htop -y # Configure Docker User sudo usermod -aG docker nicole ``` -:::caution -Be sure to change the `nicole` username in the `sudo usermod -aG docker nicole` command to whatever your local username is. -::: + +!!! warning "Warning" + Be sure to change the `nicole` username in the `sudo usermod -aG docker nicole` command to whatever your local username is. + ### Fully Logout then sign back in to the server -``` +```text exit ``` + ### Validate that permissions allow you to run docker commands while non-root -``` +```text docker ps ``` ### Initialize Minikube Cluster Additional Documentation: https://github.com/ansible/awx-operator -``` + +```text minikube start --driver=docker minikube kubectl -- get nodes minikube kubectl -- get pods -A ``` ### Make sure Minikube Cluster Automatically Starts on Boot -```jsx title="/etc/systemd/system/minikube.service" +```ini title="/etc/systemd/system/minikube.service" [Unit] Description=Minikube service After=network.target @@ -66,14 +70,15 @@ ExecStop=/usr/bin/minikube stop [Install] WantedBy=multi-user.target ``` -:::caution -Be sure to change the `nicole` username in the `User=nicole` line of the config to whatever your local username is. -::: -:::info -You can remove the `--addons=ingress` if you plan on running AWX behind an existing reverse proxy using a "**NodePort**" connection. -::: + +!!! warning "Warning" + Be sure to change the `nicole` username in the `User=nicole` line of the config to whatever your local username is. + +!!! info "Info" + You can remove the `--addons=ingress` if you plan on running AWX behind an existing reverse proxy using a "**NodePort**" connection. + ### Restart Service Daemon and Enable/Start Minikube Automatic Startup -``` +```sh sudo systemctl daemon-reload sudo systemctl enable minikube sudo systemctl start minikube @@ -81,17 +86,19 @@ sudo systemctl start minikube ### Make command alias for `kubectl` Be sure to add the following to the bottom of your existing profile file noted below. -```jsx title="~/.bashrc" + +```sh title="~/.bashrc" ... alias kubectl="minikube kubectl --" ``` -:::tip -If this is a virtual machine, now would be the best time to take a checkpoint / snapshot of the VM before moving forward, in case you need to perform rollbacks of the server(s) if you accidentally misconfigure something. -::: -## Make AWX Operator Kustomization File: +!!! tip "Tip" + If this is a virtual machine, now would be the best time to take a checkpoint / snapshot of the VM before moving forward, in case you need to perform rollbacks of the server(s) if you accidentally misconfigure something. + +## Make AWX Operator Kustomization File Find the latest tag version here: https://github.com/ansible/awx-operator/releases -```jsx title="kustomization.yml" + +```yaml title="kustomization.yml" apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: @@ -102,7 +109,8 @@ images: newTag: 2.4.0 namespace: awx ``` -```jsx title="awx.yml" + +```yaml title="awx.yml" apiVersion: awx.ansible.com/v1beta1 kind: AWX metadata: @@ -123,24 +131,31 @@ spec: selector: app.kubernetes.io/name: awx-web ``` + ### Apply Configuration File Run from the same directory as the `awx-operator.yaml` file. -``` + +```text kubectl apply -k . ``` -:::info -If you get any errors, especially ones relating to "CRD"s, wait 30 seconds, and try re-running the `kubectl apply -k .` command to fully apply the `awx.yml` configuration file to bootstrap the awx deployment. -::: + +!!! info "Info" + If you get any errors, especially ones relating to "CRD"s, wait 30 seconds, and try re-running the `kubectl apply -k .` command to fully apply the `awx.yml` configuration file to bootstrap the awx deployment. ### View Logs / Track Deployment Progress -``` +```text kubectl logs -n awx awx-operator-controller-manager -c awx-manager ``` + ### Get AWX WebUI Address -``` +```text minikube service -n awx awx-service --url ``` -### Get WebUI Password: -``` + +### Get WebUI Password +```text kubectl get secret awx-demo-admin-password -o jsonpath="{.data.password}" | base64 --decode ; echo -``` \ No newline at end of file +``` + +## Related Documentation +- [Related AWX Documentation](<../../../reference/Automation/AWX/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/automation/ansible/awx/awx.png b/deployments/automation/AWX/AWX.png similarity index 100% rename from deployments/automation/ansible/awx/awx.png rename to deployments/automation/AWX/AWX.png diff --git a/deployments/services/documentation/docusaurus.md b/deployments/automation/Documentation/Docusaurus.md similarity index 57% rename from deployments/services/documentation/docusaurus.md rename to deployments/automation/Documentation/Docusaurus.md index 00e5603..7f0742b 100644 --- a/deployments/services/documentation/docusaurus.md +++ b/deployments/automation/Documentation/Docusaurus.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: An optimized site generator in React. Docusaurus helps you to move fast and write content. Build documentation websites, blogs, marketing pages, and more. +## Purpose +An optimized site generator in React. Docusaurus helps you to move fast and write content. Build documentation websites, blogs, marketing pages, and more. ```yaml title="docker-compose.yml" version: "3" @@ -39,3 +40,7 @@ networks: ```yaml title=".env" Not Applicable ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Automation Documentation](<../../../reference/Automation/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/documentation/Material MkDocs.md b/deployments/automation/Documentation/Material MkDocs.md similarity index 89% rename from deployments/services/documentation/Material MkDocs.md rename to deployments/automation/Documentation/Material MkDocs.md index fb76893..cba177c 100644 --- a/deployments/services/documentation/Material MkDocs.md +++ b/deployments/automation/Documentation/Material MkDocs.md @@ -6,7 +6,8 @@ tags: - Docker --- -**Purpose**: Documentation that simply works. Write your documentation in Markdown and create a professional static site for your Open Source or commercial project in minutes – searchable, customizable, more than 60 languages, for all devices. +## Purpose +Documentation that simply works. Write your documentation in Markdown and create a professional static site for your Open Source or commercial project in minutes – searchable, customizable, more than 60 languages, for all devices. ## Deploy Material MKDocs ```yaml title="docker-compose.yml" @@ -37,6 +38,7 @@ N/A ## Config Example When you deploy MKDocs, you will need to give it a configuration to tell MKDocs how to structure itself. The configuration below is what I used in my deployment. This file is one folder level higher than the `/docs` folder that holds the documentation of the website. + ```yaml title="/srv/containers/material-mkdocs/docs/mkdocs.yml" # Project information site_name: Bunny Lab @@ -125,7 +127,7 @@ extra: status: new: Recently added deprecated: Deprecated - + extra_css: - stylesheets/extra.css @@ -186,3 +188,7 @@ When the server is deployed, it will come with a bunch of unnecessary documentat ## Hotloading Bug Workaround There is a [known bug](https://github.com/mkdocs/mkdocs/issues/4055) with the most recent version of Material MKDocs (as of writing) that causes it to not hotload changes immediately. This can be fixed by entering a shell in the docker container using `/bin/sh` then running the following command to downgrade the python "click" package: `pip install click==8.2.1`. After running the command, restart the container and hotloaded changes should start working again. You will have to run this command every time you re-deploy Material MKDocs until the issue is resolved officially. + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Automation Documentation](<../../../reference/Automation/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/documentation/zensical.md b/deployments/automation/Documentation/Zensical.md similarity index 58% rename from deployments/services/documentation/zensical.md rename to deployments/automation/Documentation/Zensical.md index fd7f971..458ffd3 100644 --- a/deployments/services/documentation/zensical.md +++ b/deployments/automation/Documentation/Zensical.md @@ -15,7 +15,7 @@ After many years of using Material for MKDocs and it being updated with new feat ## Architectural Overview It is useful to understand the flow of data and how everything inter-connects, so I have provided a sequence diagram that you can follow below: -``` mermaid +```mermaid sequenceDiagram autonumber actor Author as Doc Author @@ -148,6 +148,7 @@ EOF sudo systemctl daemon-reload sudo systemctl enable --now zensical-watchdog ``` + ### Updating You will obviously want to keep Zensical up-to-date. You can run the following commands to upgrade it. This is taken and simplified from the original [Upgrade Documentation](https://zensical.org/docs/upgrade/) on Zensical's website. @@ -201,142 +202,11 @@ sudo systemctl reload nginx sudo systemctl enable nginx ``` -## Gitea ACT Runner -Now is time for the arguably most-important stage of deployment, which is setting up a [Gitea Act Runner](https://docs.gitea.com/usage/actions/act-runner). This is how document changes in a Gitea repository will propagate automatically into Zensical's `/srv/zensical/docs` folder. - -```sh -# Install Dependencies -sudo apt install -y nodejs npm git rsync curl - -# Create dedicated Gitea runner service account -sudo useradd --system --create-home --home /var/lib/gitea_runner --shell /usr/sbin/nologin gitearunner || true - -# Allow the runner to write documentation changes -sudo usermod -aG zensical gitearunner - -# Allow the runner to start and stop Zensical Watchdog Service -sudo tee /etc/sudoers.d/gitearunner-systemctl > /dev/null <<'EOF' -gitearunner ALL=NOPASSWD: /usr/bin/systemctl start zensical-watchdog.service, /usr/bin/systemctl stop zensical-watchdog.service -EOF -sudo chmod 440 /etc/sudoers.d/gitearunner-systemctl -sudo chown root:root /etc/sudoers.d/gitearunner-systemctl -sudo visudo -c - -# Download Newest Gitea Runner Binary (https://gitea.com/gitea/act_runner/releases) -cd /tmp -wget https://gitea.com/gitea/act_runner/releases/download/v0.2.13/act_runner-0.2.13-linux-amd64 -sudo install -m 0755 act_runner-0.2.13-linux-amd64 /usr/local/bin/gitea_runner -gitea_runner --version - -# Generate Gitea Runner Configuration -sudo mkdir -p /etc/gitea_runner -sudo chown gitearunner:gitearunner /etc/gitea_runner -sudo -u gitearunner gitea_runner generate-config > /etc/gitea_runner/config.yaml -``` - -### Configure Registration Token -- Navigate to: "** > Settings > Actions > Runners**" - - If you don't see this, it needs to be enabled. Navigate to: "** > Settings > "Enable Repository Actions: Enabled" > Update Settings**" -- Click the "**Create New Runner**" button on the top-right of the page and copy the registration token somewhere temporarily. -- Navigate back to the GuestVM running Zensical and run the following commands. - -```sh -# Start Token Registration Process -sudo -u gitearunner env HOME=/var/lib/gitea_runner /usr/local/bin/gitea_runner register --config /etc/gitea_runner/config.yaml - -# Gitea Instance URL: https://git.bunny-lab.io -# Gitea Runner Token: -# Runner Name: zensical-docs-runner - -# Move Runner Config to Correct Location & Configure Permissions -sudo mv /tmp/.runner /var/lib/gitea_runner/.runner -sudo chown gitearunner:gitearunner /var/lib/gitea_runner/.runner -sudo chmod 600 /var/lib/gitea_runner/.runner -``` - -### Create Service -Now we need to configure the Gitea runner to start automatically via a service just like the Zensical Watchdog service. - -```sh -# Create Gitea Runner Service -sudo tee /etc/systemd/system/gitea-runner.service > /dev/null <<'EOF' -[Unit] -Description=Gitea Actions Runner (gitea_runner) -After=network-online.target -Wants=network-online.target - -[Service] -Environment=HOME=/var/lib/gitea_runner -User=gitearunner -Group=gitearunner -WorkingDirectory=/var/lib/gitea_runner -ExecStart=/usr/local/bin/gitea_runner daemon --config /etc/gitea_runner/config.yaml -Restart=always -RestartSec=2 - -[Install] -WantedBy=multi-user.target -EOF - -# Remove Container-Based Configurations to Force Runner to Run in Host Mode -sudo sed -i \ - '/^[[:space:]]*labels:/,/^[[:space:]]*cache:/{ - /^[[:space:]]*labels:/c\ labels:\n - "zensical-host:host" - /^[[:space:]]*cache:/!d - }' \ - /etc/gitea_runner/config.yaml - -# Enable and Start the Service -sudo systemctl daemon-reload -sudo systemctl enable --now gitea-runner.service -``` - -### Repository Workflow -Place the following file into your documentation repository at the given location and this will enable the runner to execute when changes happen to the repository data. - -```yaml title="gitea/workflows/automatic-deployment.yml" -name: Automatic Documentation Deployment - -on: - push: - branches: [ main ] - -jobs: - zensical_deploy: - name: Sync Docs to https://kb.bunny-lab.io - runs-on: zensical-host - - steps: - - name: Checkout Repository - uses: actions/checkout@v3 - - - name: Stop Zensical Service - run: sudo /usr/bin/systemctl stop zensical-watchdog.service - - - name: Sync repository into /srv/zensical/docs - run: | - rsync -rlD --delete \ - --exclude='.git/' \ - --exclude='.gitea/' \ - --exclude='assets/' \ - --exclude='schema/' \ - --exclude='stylesheets/' \ - --exclude='schema.json' \ - --chmod=D2775,F664 \ - . /srv/zensical/docs/ - - - name: Start Zensical Service - run: sudo /usr/bin/systemctl start zensical-watchdog.service - - - name: Notify via NTFY - if: always() - run: | - curl -d "https://kb.bunny-lab.io - Zensical job status: ${{ job.status }}" https://ntfy.bunny-lab.io/gitea-runners - -``` +## Configure Documentation Delivery +Continue with [Publish Zensical Documentation with a Host Runner](<../../../workflows/Automation/Gitea/Publish Zensical Documentation with a Host Runner.md>) to register the runner and configure repository synchronization. Return here to configure the reverse proxy. ## Traefik Reverse Proxy -It is assumed that you use a [Traefik](../edge/traefik.md) reverse proxy and are configured to use [dynamic configuration files](../edge/traefik.md#dynamic-configuration-files). Add the file below to expose the Zensical service to the rest of the world. +It is assumed that you use a [Traefik](<../../Networking and Access/Reverse Proxies/Traefik.md>) reverse proxy and are configured to use [dynamic configuration files](<../../Networking and Access/Reverse Proxies/Traefik.md#dynamic-configuration-files>). Add the file below to expose the Zensical service to the rest of the world. ```yaml title="kb.bunny-lab.io.yml" http: @@ -355,4 +225,8 @@ http: servers: - url: http://192.168.3.8:80 passHostHeader: true -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Automation Documentation](<../../../reference/Automation/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/automation/FOG Project/Configure Pi-Hole PXE Boot.md b/deployments/automation/FOG Project/Configure Pi-Hole PXE Boot.md index 161c297..65c8d34 100644 --- a/deployments/automation/FOG Project/Configure Pi-Hole PXE Boot.md +++ b/deployments/automation/FOG Project/Configure Pi-Hole PXE Boot.md @@ -1,3 +1,9 @@ +--- +tags: + - Automation + - Configure Pi-Hole PXE Boot +--- + ## Purpose This document was written to provide instructions for setting up a FOG Project server behind a Pi-Hole that has DHCP configured on it. Start by connecting to the Pi-Hole's CLI either via SSH or Portainer/Docker Interactive CLI and create the following file: @@ -20,4 +26,7 @@ dhcp-boot=undionly.kpxe,,192.168.3.9 ### Reload Pi-Hole DNS ```sh pihole reloaddns -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Automation Documentation](<../../../reference/Automation/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/automation/FOG Project/Configure Windows Server PXE Boot.md b/deployments/automation/FOG Project/Configure Windows Server PXE Boot.md index 3dd4e1d..b71d640 100644 --- a/deployments/automation/FOG Project/Configure Windows Server PXE Boot.md +++ b/deployments/automation/FOG Project/Configure Windows Server PXE Boot.md @@ -1,2 +1,15 @@ +--- +tags: + - Automation + - Configure Windows Server PXE Boot +--- + ## Purpose -This document was written to provide instructions for setting up a FOG Project server behind a Windows Server that has DHCP configured on it. \ No newline at end of file +This document was written to provide instructions for setting up a FOG Project server behind a Windows Server that has DHCP configured on it. + + +!!! warning "Incomplete Procedure" + This page records the intended DHCP/PXE integration but does not yet contain the configuration steps. It cannot be used as a complete Windows DHCP procedure. + +## Related Documentation +- [Related Automation Documentation](<../../../reference/Automation/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/automation/FOG Project/Deploy FOG Project.md b/deployments/automation/FOG Project/Deploy FOG Project.md index fb315d5..f8ce2ac 100644 --- a/deployments/automation/FOG Project/Deploy FOG Project.md +++ b/deployments/automation/FOG Project/Deploy FOG Project.md @@ -1,3 +1,9 @@ +--- +tags: + - Automation + - Deploy FOG Project +--- + ## Purpose If you want to deploy the FOG Project software in your homelab environment to remotely image devices via PXE boot, follow the instructions below to get through the process. @@ -42,9 +48,9 @@ If you want to deploy the FOG Project software in your homelab environment to re set-name: ens18 ``` - Apply the configuration safely and temporarily with `sudo netplan try` and if connectivity still works, accept the changes permanently by running `sudo netplan apply`. Otherwise, Netplan will roll back the changes automatically. + Apply the configuration safely and temporarily with `sudo netplan try` and if connectivity still works, accept the changes permanently by running `sudo netplan apply`. Otherwise, Netplan will roll back the changes automatically. -### Update & Snapshot the GuestVM +### Update and Snapshot the GuestVM Before we take a checkpoint/snapshot and install FOG Project, we want to ensure that the GuestVM is fully updated. After the updates are complete, shutdown the GuestVM, take a snapshot, and boot it back up. ```sh @@ -79,27 +85,27 @@ The steps below will guide you on a high-level through the external DHCP-server - Are you sure you wish to continue? > `Y` !!! example "Summary Example" - * Here are the settings FOG will use: - * Base Linux: Debian - * Detected Linux Distribution: Ubuntu - * Interface: ens18 - * Server IP Address: 192.168.3.9 - * Server Subnet Mask: 255.255.255.0 - * Hostname: lab-fog-01 - * Installation Type: Normal Server - * Internationalization: No - * Image Storage Location: /images - * Using FOG DHCP: No - * DHCP will NOT be setup but you must setup your + - Here are the settings FOG will use: + - Base Linux: Debian + - Detected Linux Distribution: Ubuntu + - Interface: ens18 + - Server IP Address: 192.168.3.9 + - Server Subnet Mask: 255.255.255.0 + - Hostname: lab-fog-01 + - Installation Type: Normal Server + - Internationalization: No + - Image Storage Location: /images + - Using FOG DHCP: No + - DHCP will NOT be setup but you must setup your | current DHCP server to use FOG for PXE services. - * On a Linux DHCP server you must set: next-server and filename + - On a Linux DHCP server you must set: next-server and filename - * On a Windows DHCP server you must set options 066 and 067 + - On a Windows DHCP server you must set options 066 and 067 - * Option 066/next-server is the IP of the FOG Server: (e.g. 192.168.3.9) - * Option 067/filename is the bootfile: (e.g. undionly.kkpxe or snponly.efi) - * Send OS Name, OS Version, and FOG Version: No + - Option 066/next-server is the IP of the FOG Server: (e.g. 192.168.3.9) + - Option 067/filename is the bootfile: (e.g. undionly.kkpxe or snponly.efi) + - Send OS Name, OS Version, and FOG Version: No ### Database Schema Update/Install At this point, you will be prompted to login to the website hosted by FOG Project to setup the database, navigate to the URL provided. (e.g. http://192.168.3.9/fog/management) @@ -115,186 +121,167 @@ At this point, you will be prompted to login to the website hosted by FOG Projec http://192.168.3.9/fog/management Default User Information - + - **Username**: fog - **Password**: password **Changed configurations**: The FOG installer changed configuration files and created the following backup files from your original files: - + - /etc/vsftpd.conf <=> /etc/vsftpd.conf.1777937213 - /etc/exports <=> /etc/exports.1777937213 ## Disorganized Next Steps (Under Construction) - After DHCP is correctly pointing clients to the FOG server (Options 66/67 or `next-server`/`filename`), the next phase is **PXE boot workflow + host registration and image management**—not user authentication at boot. -Short answer: **No, you do not need to create a PXE login user.** FOG does not use traditional authentication during PXE boot. Instead, it uses a **menu-driven iPXE environment** and **host registration + task assignment** from the web UI. +Short answer: **No, you do not need to create a PXE login user.** FOG does not use traditional authentication during PXE boot. Instead, it uses a **menu-driven iPXE environment** and **host registration + task assignment** from the web UI. --- ## What Actually Happens After DHCP is Set - Once DHCP is configured: -1. Client boots → gets IP + bootfile -2. Loads **iPXE from FOG** -3. Connects to FOG server -4. Displays **FOG PXE menu** -5. From there: +1. Client boots → gets IP + bootfile +2. Loads **iPXE from FOG** +3. Connects to FOG server +4. Displays **FOG PXE menu** +5. From there: - * Register host - * Deploy image - * Run tasks + - Register host + - Deploy image + - Run tasks --- ## Recommended Next Sections for Your Document - You should continue with something like this: --- ### Validate PXE Boot - Before doing anything else, confirm PXE is working end-to-end. -* Boot a test machine on the same network -* Ensure: +- Boot a test machine on the same network +- Ensure: - * It gets an IP from DHCP - * It downloads `undionly.kpxe` (BIOS) or `snponly.efi` (UEFI) - * It reaches the **FOG iPXE menu** + - It gets an IP from DHCP + - It downloads `undionly.kpxe` (BIOS) or `snponly.efi` (UEFI) + - It reaches the **FOG iPXE menu** Expected result: You see a FOG menu (not a login prompt) --- ### Register a Host in FOG - FOG needs to know about devices before imaging. #### Option 1: Register via PXE Menu (most common) - From the PXE menu: -* Select: **Perform Full Host Registration and Inventory** -* Enter: +- Select: **Perform Full Host Registration and Inventory** +- Enter: - * Hostname - * Optional description/location + - Hostname + - Optional description/location This creates a host object in FOG. #### Option 2: Pre-create in Web UI +- Go to: `FOG Management → Hosts → Create New Host` +- Add: -* Go to: `FOG Management → Hosts → Create New Host` -* Add: - - * MAC address - * Hostname + - MAC address + - Hostname --- ### Create and Capture an Image - Before deploying, you need an image stored in FOG. #### Create Image Definition +- Navigate: `Images → Create New Image` +- Set: -* Navigate: `Images → Create New Image` -* Set: + - Name + - Image Type: - * Name - * Image Type: - - * `Single Disk - Resizable` (most common) - * Storage group (default is fine) + - `Single Disk - Resizable` (most common) + - Storage group (default is fine) --- #### Assign Image to Host - -* Go to the host you registered -* Set the **Image** field to your new image +- Go to the host you registered +- Set the **Image** field to your new image --- #### Capture Image (Upload from a reference machine) - On your “golden image” machine: -1. Boot via PXE -2. Register it (if not already) -3. In Web UI: +1. Boot via PXE +2. Register it (if not already) +3. In Web UI: - * Select host - * Click **Capture** -4. Reboot machine → it uploads disk to FOG + - Select host + - Click **Capture** +4. Reboot machine → it uploads disk to FOG --- ### Deploy Image to a Host - Once an image exists: -1. Assign image to target host -2. Schedule a **Deploy Task** -3. Boot target machine via PXE +1. Assign image to target host +2. Schedule a **Deploy Task** +3. Boot target machine via PXE No login required — it will automatically start imaging if a task is assigned. --- ## Important Clarification (This Answers Your Question Directly) - > “Do we need to set up a user to login via PXE?” **No.** FOG uses: -* MAC address identification -* Task-based automation -* Optional menu interaction +- MAC address identification +- Task-based automation +- Optional menu interaction There is: -* ❌ No PXE login system by default -* ❌ No per-user authentication during boot -* ✅ Central control via the web UI +- ❌ No PXE login system by default +- ❌ No per-user authentication during boot +- ✅ Central control via the web UI --- ## Optional (Advanced Sections You Might Add Later) - If you want to make your doc more complete: ### Secure the Environment - -* Change default credentials (`fog/password`) -* Enable HTTPS (you skipped this earlier) -* Restrict PXE boot via VLANs or DHCP policies +- Change default credentials (`fog/password`) +- Enable HTTPS (you skipped this earlier) +- Restrict PXE boot via VLANs or DHCP policies ### UEFI vs BIOS Handling - -* BIOS → `undionly.kpxe` -* UEFI → `snponly.efi` +- BIOS → `undionly.kpxe` +- UEFI → `snponly.efi` ### Automating Deployments - -* Auto-assign hosts -* Use host groups -* Schedule multicast deployments +- Auto-assign hosts +- Use host groups +- Schedule multicast deployments ### Storage Optimization - -* Separate `/images` to a larger disk -* NFS tuning +- Separate `/images` to a larger disk +- NFS tuning --- ## Suggested Next Heading for Your Doc - ```md ### Validate PXE Boot and Access FOG Menu ``` @@ -303,4 +290,9 @@ followed by: ```md ### Register Hosts and Prepare Images -``` \ No newline at end of file +``` + +## Related Documentation +- [Pi-hole PXE Configuration]() — Use the documented DHCP integration when Pi-hole serves the network. +- [Windows DHCP PXE Scaffold]() — Check the explicitly incomplete alternative before choosing it. +- [Related Automation Documentation](<../../../reference/Automation/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/devops/gitea.md b/deployments/automation/Gitea/Gitea.md similarity index 75% rename from deployments/services/devops/gitea.md rename to deployments/automation/Gitea/Gitea.md index 7f0ee19..5ce5f84 100644 --- a/deployments/services/devops/gitea.md +++ b/deployments/automation/Gitea/Gitea.md @@ -5,11 +5,11 @@ tags: - Docker --- -**Purpose**: Gitea is a painless self-hosted all-in-one software development service, it includes Git hosting, code review, team collaboration, package registry and CI/CD. It is similar to GitHub, Bitbucket and GitLab. Gitea was forked from Gogs originally and almost all the code has been changed. +## Purpose +Gitea is a painless self-hosted all-in-one software development service, it includes Git hosting, code review, team collaboration, package registry and CI/CD. It is similar to GitHub, Bitbucket and GitLab. Gitea was forked from Gogs originally and almost all the code has been changed. [Detailed SMTP Configuration Reference](https://docs.gitea.com/administration/config-cheat-sheet) - ## Docker Configuration ```yaml title="docker-compose.yml" version: "3" @@ -81,6 +81,7 @@ POSTGRES_PASSWORD=SomethingSuperSecure ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -99,4 +100,9 @@ http: servers: - url: http://192.168.5.70:3000 passHostHeader: true -``` \ No newline at end of file +``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Gitea Workflows](<../../../reference/Automation/Gitea Configuration Delivery.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/automation/puppet/deployment/Puppet Bolt.md b/deployments/automation/Puppet/Puppet Bolt.md similarity index 88% rename from deployments/automation/puppet/deployment/Puppet Bolt.md rename to deployments/automation/Puppet/Puppet Bolt.md index 7fc6442..45337d9 100644 --- a/deployments/automation/puppet/deployment/Puppet Bolt.md +++ b/deployments/automation/Puppet/Puppet Bolt.md @@ -4,13 +4,14 @@ tags: - Automation --- -**Purpose**: Puppet Bolt can be leveraged in an Ansible-esque manner to connect to and enroll devices such as Windows Servers, Linux Servers, and various workstations. To this end, it could be used to run ad-hoc tasks or enroll devices into a centralized Puppet server. (e.g. `LAB-PUPPET-01.bunny-lab.io`) +## Purpose +Puppet Bolt can be leveraged in an Ansible-esque manner to connect to and enroll devices such as Windows Servers, Linux Servers, and various workstations. To this end, it could be used to run ad-hoc tasks or enroll devices into a centralized Puppet server. (e.g. `LAB-PUPPET-01.bunny-lab.io`) !!! note "Assumptions" - This deployment assumes you are deploying Puppet bolt onto the same server as Puppet. If you have not already, follow the [Puppet Deployment](./puppet.md) documentation to do so before continuing with the Puppet Bolt deployment. + This deployment assumes you are deploying Puppet bolt onto the same server as Puppet. If you have not already, follow the [Puppet Deployment]() documentation to do so before continuing with the Puppet Bolt deployment. ## Initial Preparation -``` sh +```sh # Install Bolt Repository sudo rpm -Uvh https://yum.puppet.com/puppet-tools-release-el-9.noarch.rpm sudo yum install -y puppet-bolt @@ -32,7 +33,8 @@ bolt project init bunny_lab ## Configuring Inventory At this point, you will want to create an inventory file that you can use for tracking devices. For now, this will have hard-coded credentials until a cleaner method is figured out. -``` yaml title="/etc/puppetlabs/bolt/inventory.yaml" + +```yaml title="/etc/puppetlabs/bolt/inventory.yaml" # Inventory file for Puppet Bolt groups: - name: linux_servers @@ -78,14 +80,16 @@ groups: ### Validate Bolt Inventory Works If the inventory file is created correctly, you will see the hosts listed when you run the command below: -``` sh + +```sh cd /etc/puppetlabs/bolt bolt inventory show ``` ??? example "Example Output of `bolt inventory show`" You should expect to see output similar to the following: - ``` sh + + ```sh [root@lab-puppet-01 bolt-lab]# bolt inventory show Targets lab-auth-01.bunny-lab.io @@ -118,25 +122,26 @@ If you work with Windows-based devices in a domain environment, you will need to We need to install the necessary software on the puppet server to allow Kerberos authentication to occur. === "Rocky, CentOS, RHEL, Fedora" - ``` sh + ```sh sudo yum install krb5-workstation ``` - + === "Debian, Ubuntu" - ``` sh + ```sh sudo apt-get install krb5-user ``` === "SUSE" - ``` sh + ```sh sudo zypper install krb5-client ``` ### Prepare `/etc/krb5.conf` Configuration We need to configure Kerberos to know how to reach the domain, this is achieved by editing `/etc/krb5.conf` to look similar to the following, with your own domain substituting the example values. -``` ini + +```ini [libdefaults] default_realm = BUNNY-LAB.IO dns_lookup_realm = false @@ -162,14 +167,16 @@ We need to configure Kerberos to know how to reach the domain, this is achieved ### Initialize Kerberos Connection Now we need to log into the domain using (preferrably) domain administrator credentials, such as the example below. You will be prompted to enter your domain password. -``` sh + +```sh kinit nicole.rappe@BUNNY-LAB.IO klist ``` ??? example "Example Output of `klist`" You should expect to see output similar to the following. Finding a way to ensure the Kerberos tickets live longer is still under research, as 7 days is not exactly practical for long-term deployments. - ``` sh + + ```sh [root@lab-puppet-01 bolt-lab]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: nicole.rappe@BUNNY-LAB.IO @@ -182,20 +189,21 @@ klist ### Prepare Windows Devices Windows devices need to be prepared ahead-of-time in order for WinRM functionality to work as-expected. I have prepared a powershell script that you can run on each device that needs remote management functionality. You can port this script based on your needs, and deploy it via whatever methods you have available to you. (e.g. Ansible, Group Policies, existing RMM software, manually via remote desktop, etc). -You can find the [WinRM Enablement Script](../../../../workflows/operations/automation/ansible/enable-winrm-on-windows-devices.md) in the Bunny Lab documentation. +You can find the [WinRM Enablement Script](<../../../workflows/Identity and Certificates/Windows/Enable WinRM over HTTPS.md>) in the Bunny Lab documentation. ## Ad-Hoc Command Examples At this point, you should finally be ready to connect to Windows and Linux devices and run commands on them ad-hoc. Puppet Bolt Modules and Plans will be discussed further down the road. ??? example "Example Output of `bolt command run whoami -t domain_controllers --no-ssl-verify`" You should expect to see output similar to the following. This is what you will see when leveraging WinRM via Kerberos on Windows devices. - ``` sh + + ```sh [root@lab-puppet-01 bolt-lab]# bolt command run whoami -t domain_controllers --no-ssl-verify CLI arguments ["ssl-verify"] might be overridden by Inventory: /tmp/bolt-lab/inventory.yaml [ID: cli_overrides] Started on lab-dc-01.bunny-lab.io... Started on lab-dc-02.bunny-lab.io... Finished on lab-dc-02.bunny-lab.io: - bunny-lab\nicole.rappe + bunny-lab\nicole.rappe Finished on lab-dc-01.bunny-lab.io: bunny-lab\nicole.rappe Successful on 2 targets: lab-dc-01.bunny-lab.io,lab-dc-02.bunny-lab.io @@ -204,16 +212,19 @@ At this point, you should finally be ready to connect to Windows and Linux devic ??? example "Example Output of `bolt command run whoami -t linux_servers`" You should expect to see output similar to the following. This is what you will see when leveraging native SSH on Linux devices. - ``` sh + + ```sh [root@lab-puppet-01 bolt-lab]# bolt command run whoami -t linux_servers CLI arguments ["ssl-verify"] might be overridden by Inventory: /tmp/bolt-lab/inventory.yaml [ID: cli_overrides] Started on lab-auth-01.bunny-lab.io... Started on lab-auth-02.bunny-lab.io... Finished on lab-auth-02.bunny-lab.io: - nicole + nicole Finished on lab-auth-01.bunny-lab.io: nicole Successful on 2 targets: lab-auth-01.bunny-lab.io,lab-auth-02.bunny-lab.io Ran on 2 targets in 0.68 sec ``` +## Related Documentation +- [Related Automation Documentation](<../../../reference/Automation/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/automation/puppet/deployment/puppet.md b/deployments/automation/Puppet/Puppet.md similarity index 92% rename from deployments/automation/puppet/deployment/puppet.md rename to deployments/automation/Puppet/Puppet.md index e216fcd..6f0b87a 100644 --- a/deployments/automation/puppet/deployment/puppet.md +++ b/deployments/automation/Puppet/Puppet.md @@ -4,15 +4,15 @@ tags: - Automation --- -**Purpose**: -Puppet is another declarative configuration management tool that excels in system configuration and enforcement. Like Ansible, it's designed to maintain the desired state of a system's configuration but uses a client-server (master-agent) architecture by default. +## Purpose +Puppet is another declarative configuration management tool that excels in system configuration and enforcement. Like Ansible, it's designed to maintain the desired state of a system's configuration but uses a client-server (master-agent) architecture by default. !!! note "Assumptions" This document assumes you are deploying Puppet server onto Rocky Linux 9.4. Any version of RHEL/CentOS/Alma/Rocky should behave similarily. ## Architectural Overview ### Detailed -``` mermaid +```mermaid sequenceDiagram participant Gitea as Gitea Repo (Puppet Environment) participant r10k as r10k (Environment Deployer) @@ -33,24 +33,24 @@ sequenceDiagram Agent->>PuppetMaster: Request to enroll (Agent Check-in) PuppetMaster->>Agent: Verify SSL Certificate & Authenticate Agent-->>PuppetMaster: Send facts about system (Facter) - + %% PuppetMaster compiles catalog for the agent PuppetMaster->>PuppetMaster: Compile Catalog PuppetMaster->>PuppetMaster: Check if 'neofetch' is required in manifest PuppetMaster-->>Agent: Send compiled catalog with 'neofetch' installation instructions - + %% Agent installs neofetch Agent->>Agent: Check if 'neofetch' is installed Agent--xNeofetch: 'neofetch' not installed Agent->>Neofetch: Install 'neofetch' Neofetch-->>Agent: Installation complete - + %% Agent reports back to PuppetMaster Agent->>PuppetMaster: Report status (catalog applied and neofetch installed) ``` ### Simplified -``` mermaid +```mermaid sequenceDiagram participant Gitea as Gitea (Puppet Repository) participant PuppetMaster as Puppet Server @@ -60,7 +60,7 @@ sequenceDiagram %% PuppetMaster pulling environment updates PuppetMaster->>Gitea: Pull environment updates Gitea-->>PuppetMaster: Send updated code - + %% Agent enrollment and catalog request Agent->>PuppetMaster: Request catalog (Check-in) PuppetMaster->>Agent: Send compiled catalog (neofetch required) @@ -68,43 +68,45 @@ sequenceDiagram %% Agent installs neofetch Agent->>Neofetch: Install neofetch Neofetch-->>Agent: Installation complete - + %% Agent reports back Agent->>PuppetMaster: Report catalog applied (neofetch installed) ``` ### Breakdown #### 1. **PuppetMaster Pulls Updates from Gitea** - - PuppetMaster uses `r10k` to fetch the latest environment updates from Gitea. These updates include manifests, hiera data, and modules for the specified Puppet environments. + - PuppetMaster uses `r10k` to fetch the latest environment updates from Gitea. These updates include manifests, hiera data, and modules for the specified Puppet environments. #### 2. **PuppetMaster Compiles Catalogs and Modules** - - After pulling updates, the PuppetMaster compiles the latest node-specific catalogs based on the manifests and modules. It ensures the configuration is ready for agents to retrieve. + - After pulling updates, the PuppetMaster compiles the latest node-specific catalogs based on the manifests and modules. It ensures the configuration is ready for agents to retrieve. #### 3. **Agent (fedora.bunny-lab.io) Checks In** - - The Puppet agent on `fedora.bunny-lab.io` checks in with the PuppetMaster for its catalog. This request tells the PuppetMaster to compile the node's desired configuration. + - The Puppet agent on `fedora.bunny-lab.io` checks in with the PuppetMaster for its catalog. This request tells the PuppetMaster to compile the node's desired configuration. #### 4. **Agent Downloads and Applies the Catalog** - - The agent retrieves its compiled catalog from the PuppetMaster. It compares the current system state with the desired state outlined in the catalog. + - The agent retrieves its compiled catalog from the PuppetMaster. It compares the current system state with the desired state outlined in the catalog. #### 5. **Agent Installs `neofetch`** - - The agent identifies that `neofetch` is missing and installs it using the system's package manager. The installation follows the directives in the catalog. + - The agent identifies that `neofetch` is missing and installs it using the system's package manager. The installation follows the directives in the catalog. #### 6. **Agent Reports Success** - - Once changes are applied, the agent sends a report back to the PuppetMaster. The report includes details of the changes made, confirming `neofetch` was installed. + - Once changes are applied, the agent sends a report back to the PuppetMaster. The report includes details of the changes made, confirming `neofetch` was installed. -## Deployment Steps: +## Deployment Steps You will need to perform a few steps outlined in the [official Puppet documentation](https://www.puppet.com/docs/puppet/7/install_puppet.html) to get a Puppet server operational. A summarized workflow is seen below: ### Install Puppet Repository **Installation Scope**: Puppet Server / Managed Devices -``` sh + +```sh # Add Puppet Repository / Enable Puppet on YUM sudo rpm -Uvh https://yum.puppet.com/puppet7-release-el-9.noarch.rpm ``` ### Install Puppet Server **Installation Scope**: Puppet Server -``` sh + +```sh # Install the Puppet Server sudo yum install -y puppetserver systemctl enable --now puppetserver @@ -116,7 +118,8 @@ puppetserver -v ### Install Puppet Agent **Installation Scope**: Puppet Server / Managed Devices -``` sh + +```sh # Install Puppet Agent (This will already be installed on the Puppet Server) sudo yum install -y puppet-agent @@ -138,13 +141,15 @@ puppetserver ca sign --certname fedora.bunny-lab.io #### Validate Agent Functionality At this point, you want to ensure that the device being managed by the agent is able to pull down configurations from the Puppet Server. You will know if it worked by getting a message similar to `Notice: Applied catalog in X.XX seconds` after running the following command: -``` sh + +```sh puppet agent --test ``` ## Install r10k -At this point, we need to configure Gitea as the storage repository for the Puppet "Environments" (e.g. `Production` and `Development`). We can do this by leveraging a tool called "r10k" which pulls a Git repository and configures it as the environment in Puppet. -``` sh +At this point, we need to configure Gitea as the storage repository for the Puppet "Environments" (e.g. `Production` and `Development`). We can do this by leveraging a tool called "r10k" which pulls a Git repository and configures it as the environment in Puppet. + +```sh # Install r10k Pre-Requisites sudo dnf install -y ruby ruby-devel gcc make @@ -157,7 +162,7 @@ r10k version ``` ### Configure r10k -``` sh +```sh # Create the r10k Configuration Directory sudo mkdir -p /etc/puppetlabs/r10k @@ -177,7 +182,7 @@ sources: basedir: '/etc/puppetlabs/code/environments' ``` -``` sh +```sh # Lockdown the Permissions of the Configuration File sudo chmod 600 /etc/puppetlabs/r10k/r10k.yaml @@ -194,7 +199,7 @@ You will make a repository on Gitea with the following files and structure as no !!! example "Example Agent & Neofetch" You will notice there is a section for `fedora.bunny-lab.io` as well as mentions of `neofetch`. These are purely examples in my homelab of a computer I was testing against during the development of the Puppet Server and associated documentation. You can feel free to not include the entire `modules/neofetch/manifests/init.pp` file in the Gitea repository, as well as remove this entire section from the `manifests/site.pp` file: - ``` yaml + ```yaml # Node definition for the Fedora agent node 'fedora.bunny-lab.io' { # Include the neofetch class to ensure Neofetch is installed @@ -215,7 +220,7 @@ You will make a repository on Gitea with the following files and structure as no mod 'puppetlabs-concat', '9.0.2' mod 'puppet-systemd', '7.1.0' ``` - + === "environment.conf" This file is mostly redundant, as it states the values below, which are the default values Puppet works with. I only included it in case I had a unique use-case that required a more custom approach to the folder structure. (This is very unlikely). @@ -238,7 +243,8 @@ You will make a repository on Gitea with the following files and structure as no !!! example "Neofetch Example" In the example configuration below, you will notice this section. This tells Puppet to deploy the neofetch package to any device that has `include neofetch` written. Grouping devices etc is currently undocumented as of writing this. - ``` sh + + ```sh # Node definition for the Fedora agent node 'fedora.bunny-lab.io' { # Include the neofetch class to ensure Neofetch is installed @@ -312,7 +318,8 @@ Navigate to "**Gitea > User (Top-Right) > Settings > Applications It is critical that you store the token somewhere safe like a password manager as you will need to reference it later and might need it in the future if you re-build the r10k environment. Now we want to configure Gitea to store the credentials for later use by r10k: -``` sh + +```sh # Enable Stored Credentials (We will address security concerns further down...) sudo yum install -y git sudo git config --global credential.helper store @@ -333,7 +340,8 @@ sudo rm -rf /tmp/PuppetTest ``` Finally we validate that everything is working by pulling down the Puppet environments using r10k on the Puppet Server: -``` sh + +```sh # Deploy Puppy Environments from Gitea sudo /usr/local/bin/r10k deploy environment -p @@ -342,11 +350,11 @@ sudo ls /etc/puppetlabs/code/environments/production/modules sudo ls /etc/puppetlabs/code/environments/development/modules ``` -!!! success "Successful Puppet Environment Deployment +!!! success "Successful Puppet Environment Deployment" If you got no errors about Puppetfile formatting or Gitea permissions errors, then you are good to move onto the next step. ## External Node Classifier (ENC) -An ENC allows you to define node-specific data, including the environment, on the Puppet Server. The agent requests its configuration, and the Puppet Server provides the environment and classes to apply. +An ENC allows you to define node-specific data, including the environment, on the Puppet Server. The agent requests its configuration, and the Puppet Server provides the environment and classes to apply. **Advantages**: @@ -355,7 +363,7 @@ An ENC allows you to define node-specific data, including the environment, on th - **Scalability**: Suitable for managing environments for hundreds or thousands of nodes. ### Create an ENC Script -``` sh +```sh sudo mkdir -p /opt/puppetlabs/server/data/puppetserver/scripts/ ``` @@ -392,13 +400,14 @@ output = { puts output.to_yaml ``` -``` sh +```sh # Ensure the File is Executable sudo chmod +x /opt/puppetlabs/server/data/puppetserver/scripts/enc.rb ``` ### Configure Puppet Server to Use the ENC Edit the Puppet Server's `puppet.conf` and set the `node_terminus` and `external_nodes` parameters: + ```ini title="/etc/puppetlabs/puppet/puppet.conf" [master] node_terminus = exec @@ -406,13 +415,15 @@ external_nodes = /opt/puppetlabs/server/data/puppetserver/scripts/enc.rb ``` Restart the Puppet Service -``` sh + +```sh sudo systemctl restart puppetserver ``` ## Pull Puppet Environments from Gitea At this point, we can tell r10k to pull down the Puppet environments (e.g. `Production` and `Development`) that we made in the Gitea repository in previous steps. Run the following command on the Puppet Server to pull down the environments. This will download / configure any Puppet Forge modules as well as any hand-made modules such as Neofetch. -``` sh + +```sh sudo /usr/local/bin/r10k deploy environment -p # OPTIONAL: You can pull down a specific environment instead of all environments if you specify the branch name, seen here: #sudo /usr/local/bin/r10k deploy environment development -p @@ -420,9 +431,13 @@ sudo /usr/local/bin/r10k deploy environment -p ### Apply Configuration to Puppet Server At this point, we are going to deploy the configuration from Gitea to the Puppet Server itself so it installs PuppetDB automatically as well as configures firewall ports and other small things to functional properly. Once this is completed, you can add additional agents / managed devices and they will be able to communicate with the Puppet Server over the network. -``` sh + +```sh sudo /opt/puppetlabs/bin/puppet agent -t ``` !!! success "Puppet Server Deployed and Validated" - Congradulations! You have successfully deployed an entire Puppet Server, as well as integrated Gitea and r10k to deploy environment changes in a versioned environment, as well as validated functionality against a managed device using the agent (such as a spare laptop/desktop). If you got this far, be proud, because it took me over 12 hours write this documentation allowing you to deploy a server in less than 30 minutes. \ No newline at end of file + Congradulations! You have successfully deployed an entire Puppet Server, as well as integrated Gitea and r10k to deploy environment changes in a versioned environment, as well as validated functionality against a managed device using the agent (such as a spare laptop/desktop). If you got this far, be proud, because it took me over 12 hours write this documentation allowing you to deploy a server in less than 30 minutes. + +## Related Documentation +- [Related Automation Documentation](<../../../reference/Automation/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/rmm/tacticalrmm.md b/deployments/automation/Remote Management/TacticalRMM.md similarity index 74% rename from deployments/services/rmm/tacticalrmm.md rename to deployments/automation/Remote Management/TacticalRMM.md index 52dc6cb..63ead45 100644 --- a/deployments/services/rmm/tacticalrmm.md +++ b/deployments/automation/Remote Management/TacticalRMM.md @@ -4,14 +4,14 @@ tags: - RMM --- -**Purpose**: +## Purpose Tactical RMM is a remote monitoring & management tool built with Django, Vue and Golang. [Official Documentation](https://docs.tacticalrmm.com/install_server/). -!!! Requirements - Ubuntu Server 22.04 LTS, 8GB RAM, 64GB Storage. +!!! info "Requirements" + Ubuntu Server 22.04 LTS, 8GB RAM, 64GB Storage. ## Deployment Script -``` +```sh # Check for Updates sudo apt update sudo apt install -y wget curl sudo ufw @@ -36,4 +36,7 @@ sudo su - tactical wget https://raw.githubusercontent.com/amidaware/tacticalrmm/master/install.sh chmod +x install.sh ./install.sh -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Automation Documentation](<../../../reference/Automation/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/Automation Tools/Activepieces.md b/deployments/automation/Tools/Activepieces.md similarity index 77% rename from deployments/services/Automation Tools/Activepieces.md rename to deployments/automation/Tools/Activepieces.md index 63350f8..6f89bc9 100644 --- a/deployments/services/Automation Tools/Activepieces.md +++ b/deployments/automation/Tools/Activepieces.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: Self-hosted open-source no-code business automation tool. +## Purpose +Self-hosted open-source no-code business automation tool. ```yaml title="docker-compose.yml" version: '3.0' @@ -40,7 +41,7 @@ services: - /srv/containers/activepieces/postgresql:/var/lib/postgresql/data' networks: docker_network: - ipv4_address: 192.168.5.61 + ipv4_address: 192.168.5.61 redis: image: 'redis:7.0.7' container_name: redis @@ -49,7 +50,7 @@ services: - /srv/containers/activepieces/redis:/data' networks: docker_network: - ipv4_address: 192.168.5.60 + ipv4_address: 192.168.5.60 networks: default: external: @@ -58,7 +59,7 @@ networks: external: true ``` -```jsx title=".env" +```ini title=".env" AP_ENGINE_EXECUTABLE_PATH=dist/packages/engine/main.js AP_ENCRYPTION_KEY=e81f8754faa04acaa7b13caa5d2c6a5a AP_JWT_SECRET=REDACTED #BE SURE TO SET THIS WITH A VALID JWT SECRET > REFER TO OFFICIAL DOCUMENTATION @@ -75,3 +76,7 @@ AP_REDIS_PORT=6379 AP_SANDBOX_RUN_TIME_SECONDS=600 AP_TELEMETRY_ENABLED=true ``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Automation Documentation](<../../../reference/Automation/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/automation/Tools/Node Red.md b/deployments/automation/Tools/Node Red.md new file mode 100644 index 0000000..ccf7f1d --- /dev/null +++ b/deployments/automation/Tools/Node Red.md @@ -0,0 +1,41 @@ +--- +tags: + - Node-RED + - Automation + - Docker +--- + +## Purpose +Node-RED is a programming tool for wiring together hardware devices, APIs and online services in new and interesting ways. + +```yaml title="docker-compose.yml" +version: "3.7" + +services: + node-red: + image: nodered/node-red:latest + environment: + - TZ=America/Denver + ports: + - "1880:1880" + networks: + docker_network: + ipv4_address: 192.168.5.92 + volumes: + - /srv/containers/node-red:/data +networks: + default: + external: + name: docker_network + docker_network: + external: true + +``` + +```yaml title=".env" +Not Applicable +``` + +## Related Documentation +- [Docker Network Prerequisite](<../../Containers/Docker/Create the Docker Network.md>) — The configuration references the external `docker_network`; prepare it on the intended Docker host. +- [Related Automation Documentation](<../../../reference/Automation/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/deployments/services/Automation Tools/Semaphore UI.md b/deployments/automation/Tools/Semaphore UI.md similarity index 82% rename from deployments/services/Automation Tools/Semaphore UI.md rename to deployments/automation/Tools/Semaphore UI.md index 94b237a..adf3048 100644 --- a/deployments/services/Automation Tools/Semaphore UI.md +++ b/deployments/automation/Tools/Semaphore UI.md @@ -5,7 +5,8 @@ tags: - Docker --- -**Purpose**: User friendly web interface for executing Ansible playbooks, Terraform, OpenTofu code and Bash scripts. It is designed to make your automation tasks easier and more enjoyable. +## Purpose +User friendly web interface for executing Ansible playbooks, Terraform, OpenTofu code and Bash scripts. It is designed to make your automation tasks easier and more enjoyable. [Website Details](https://semaphoreui.com/) @@ -42,7 +43,7 @@ services: - /srv/containers/semaphore-ui/tmp:/tmp/semaphore depends_on: - postgres - + postgres: image: postgres:12-alpine ports: @@ -63,6 +64,7 @@ N/A - Will be cleaned up later. ## Traefik Reverse Proxy Configuration If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. + ```yaml http: routers: @@ -81,4 +83,8 @@ http: servers: - url: http://192.168.3.51:3000 passHostHeader: true -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Automation Documentation](<../../../reference/Automation/index.md>) — Find the connected deployments, procedures, and references for this subject. +- [Traefik Deployment](<../../Networking and Access/Reverse Proxies/Traefik.md>) — Prepare the reverse proxy before applying this page's routing configuration. diff --git a/deployments/automation/index.md b/deployments/automation/index.md index 514f4ad..b35297d 100644 --- a/deployments/automation/index.md +++ b/deployments/automation/index.md @@ -1,38 +1,22 @@ --- tags: - - Operations - Automation - - Index + - Deployments - Documentation --- # Automation ## Purpose -Infrastructure automation, orchestration, and workflow tooling. +Find deployments for automation. Follow the subject guide to choose the relevant environment and connect this material to the other document types. ## Includes -- Ansible and Puppet patterns -- Inventory and credential conventions -- CI/CD and automation notes +- AWX +- Documentation +- FOG Project +- Gitea +- Puppet +- Remote Management +- Tools -## New Document Template -````markdown -# -## Purpose - - -!!! info "Assumptions" - - - - - -## Inputs -- - -## Procedure -```sh -# Commands or job steps -``` - -## Validation -- -```` +## Follow the Subject +[Automation](<../../reference/Automation/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/deployments/index.md b/deployments/index.md index 48066b0..deab7f7 100644 --- a/deployments/index.md +++ b/deployments/index.md @@ -1,15 +1,32 @@ --- tags: - Deployments - - Index - Documentation --- # Deployments ## Purpose -Build and deployment documentation for platforms, services, and automation stacks. +Build the system you need, then follow its linked maintenance and reference material. Use the subject guides to select prerequisites and distinguish documented implementation alternatives. ## Includes -- Platform deployments (virtualization and containerization) -- Service deployments and integration patterns -- Automation stack deployment guides +- Applications +- Automation +- Backup and Recovery +- Containers +- Identity and Certificates +- Networking and Access +- Virtualization and Storage +- Windows and Linux + +## Start with a Subject +- [Applications](<../reference/Applications/index.md>) — Find applications by the service they provide, then continue to their deployment, authentication, data, and maintenance documentation. +- [Automation](<../reference/Automation/index.md>) — Connect source control, automation controllers, managed hosts, and configuration delivery. Use the documented execution environment and authentication method for each workflow. +- [Backup and Recovery](<../reference/Backup and Recovery/index.md>) — Find backup concepts, repository maintenance, and recovery dependencies. Select the procedure for the affected backup system and distinguish a backup restore from a replica or snapshot operation. +- [Containers](<../reference/Containers/index.md>) — Prepare the Docker or Kubernetes environment used by application deployments, then follow the operating procedures for building, moving, and exposing workloads. +- [Identity and Certificates](<../reference/Identity and Certificates/index.md>) — Connect directory services, certificate trust, single sign-on, and application authentication. Start with the identity system involved, then follow the integration or maintenance procedure. +- [Networking and Access](<../reference/Networking and Access/index.md>) — Find the DNS, proxy, VPN, and remote-access instructions that connect users and services. Use the address plans to identify the intended network before changing connectivity. +- [Virtualization and Storage](<../reference/Virtualization and Storage/index.md>) — Follow the relationship between hypervisors, shared storage, guest disks, and recovery procedures. Select the documented storage design before choosing a maintenance command. +- [Windows and Linux](<../reference/Windows and Linux/index.md>) — Find workstation and server operating-system setup, updates, and repairs. Storage, networking, and identity tasks are linked to their subject guides when they cross operating-system boundaries. + +## Find Related Knowledge +[The subject guides](<../reference/index.md>) connect these deployments to the other document roles. diff --git a/deployments/platforms/containerization/Docker/Custom Containers/Git Repo Updater.md b/deployments/platforms/containerization/Docker/Custom Containers/Git Repo Updater.md deleted file mode 100644 index 0b82a38..0000000 --- a/deployments/platforms/containerization/Docker/Custom Containers/Git Repo Updater.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -tags: - - Containers - - Docker - - Containerization ---- - -**Purpose**: Docker container running Alpine Linux that automates and improves upon much of the script mentioned in the [Git Repo Updater](../../../../../scripts/bash/git-repo-updater.md) document. It offers the additional benefits of checking for updates every 5 seconds instead of every 60 seconds. It also accepts environment variables to provide credentials and notification settings, and can have an infinite number of monitored repositories. - -### Deployment -You can find the current up-to-date Gitea repository that includes the `docker-compose.yml` and `.env` files that you need to deploy everything [here](https://git.bunny-lab.io/container-registry/-/packages/container/git-repo-updater/latest) -```jsx title="docker-compose.yml" -version: '3.3' -services: - git-repo-updater: - privileged: true - container_name: git-repo-updater - env_file: - - stack.env - image: git.bunny-lab.io/container-registry/git-repo-updater:latest - volumes: - - /srv/containers:/srv/containers - - /srv/containers/git-repo-updater/Repo_Cache:/root/Repo_Cache - restart: always -``` - -```jsx title=".env" -# Gitea Credentials -GIT_USERNAME=nicole.rappe -GIT_PASSWORD=USE-AN-APP-PASSWORD - -# NTFY Push Notification Server URL -NTFY_URL=https://ntfy.cyberstrawberry.net/git-repo-updater - -# Repository/Destination Pairs (Add as Many as Needed) -REPO_01="https://${GIT_USERNAME}:${GIT_PASSWORD}@git.bunny-lab.io/bunny-lab/docs.git,/srv/containers/material-mkdocs/docs/docs" -REPO_02="https://${GIT_USERNAME}:${GIT_PASSWORD}@git.bunny-lab.io/GitOps/servers.bunny-lab.io.git,/srv/containers/homepage-docker" -``` -### Build / Development -If you want to learn how the container was assembled, the related build files are located [here](https://git.cyberstrawberry.net/container-registry/git-repo-updater) -```jsx title="Dockerfile" -# Use Alpine as the base image of the container -FROM alpine:latest - -# Install necessary packages -RUN apk --no-cache add git curl rsync - -# Add script -COPY repo_watcher.sh /repo_watcher.sh -RUN chmod +x /repo_watcher.sh - -#Create Directory to store Repositories -RUN mkdir -p /root/Repo_Cache - -# Start script (Alpine uses /bin/sh instead of /bin/bash) -CMD ["/bin/sh", "-c", "/repo_watcher.sh"] -``` - -```jsx title="repo_watcher.sh" -#!/bin/sh - -# Function to process each repo-destination pair -process_repo() { - FULL_REPO_URL=$1 - DESTINATION=$2 - - # Extract the URL without credentials for logging and notifications - CLEAN_REPO_URL=$(echo "$FULL_REPO_URL" | sed 's/https:\/\/[^@]*@/https:\/\//') - - # Directory to hold the repository locally - REPO_DIR="/root/Repo_Cache/$(basename $CLEAN_REPO_URL .git)" - - # Clone the repo if it doesn't exist, or navigate to it if it does - if [ ! -d "$REPO_DIR" ]; then - curl -d "Cloning: $CLEAN_REPO_URL" $NTFY_URL - git clone "$FULL_REPO_URL" "$REPO_DIR" > /dev/null 2>&1 - fi - cd "$REPO_DIR" || exit - - # Fetch the latest changes - git fetch origin main > /dev/null 2>&1 - - # Check if the local repository is behind the remote - LOCAL=$(git rev-parse @) - REMOTE=$(git rev-parse @{u}) - - if [ "$LOCAL" != "$REMOTE" ]; then - curl -d "Updating: $CLEAN_REPO_URL" $NTFY_URL - git pull origin main > /dev/null 2>&1 - rsync -av --delete --exclude '.git/' ./ "$DESTINATION" > /dev/null 2>&1 - fi -} - -# Main loop -while true; do - # Iterate over each environment variable matching 'REPO_[0-9]+' - env | grep '^REPO_[0-9]\+=' | while IFS='=' read -r name value; do - # Split the value by comma and read into separate variables - OLD_IFS="$IFS" # Save the original IFS - IFS=',' # Set IFS to comma for splitting - set -- $value # Set positional parameters ($1, $2, ...) - REPO_URL="$1" # Assign first parameter to REPO_URL - DESTINATION="$2" # Assign second parameter to DESTINATION - IFS="$OLD_IFS" # Restore original IFS - - process_repo "$REPO_URL" "$DESTINATION" - done - - # Wait for 5 seconds before the next iteration - sleep 5 -done - -``` - diff --git a/deployments/platforms/index.md b/deployments/platforms/index.md deleted file mode 100644 index 1ca4322..0000000 --- a/deployments/platforms/index.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -tags: - - Platforms - - Index - - Documentation ---- - -# Platforms -## Purpose -Virtualization and containerization platforms, cluster builds, and base OS images. - -## Includes -- Hypervisors and virtualization stacks -- Kubernetes and Docker foundations -- Base image and cluster provisioning patterns - -## New Document Template -````markdown -# -## Purpose - - -!!! info "Assumptions" - - - - - -## Architectural Overview - - -## Procedure -```sh -# Commands (grouped and annotated) -``` - -## Validation -- -```` diff --git a/deployments/platforms/virtualization/Rancher Harvester/Harvester.md b/deployments/platforms/virtualization/Rancher Harvester/Harvester.md deleted file mode 100644 index 20d5453..0000000 --- a/deployments/platforms/virtualization/Rancher Harvester/Harvester.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -tags: - - Rancher - - Harvester ---- - -**Purpose**: Rancher Harvester is an awesome tool that acts like a self-hosted cloud VDI provider, similar to AWS, Linode, and other online cloud compute platforms. In most scenarios, you will deploy "Rancher" in addition to Harvester to orchestrate the deployment, management, and rolling upgrades of a Kubernetes Cluster. You can also just run standalone Virtual Machines, similar to Hyper-V, RHEV, oVirt, Bhyve, XenServer, XCP-NG, and VMware ESXi. - -:::note Prerequisites -This document assumes your bare-metal host has at least 32GB of Memory, 200GB of Disk Space, and 8 processor cores. See [Recommended System Requirements](https://docs.harvesterhci.io/v1.1/install/requirements) -::: - -## First Harvester Node -### Download Installer ISO -You will need to navigate to the Rancher Harvester GitHub to download the [latest ISO release of Harvester](https://releases.rancher.com/harvester/v1.1.2/harvester-v1.1.2-amd64.iso), currently **v1.1.2**. Then image it onto a USB flashdrive using a tool like [Rufus](https://github.com/pbatard/rufus/releases/download/v4.2/rufus-4.2p.exe). Proceed to boot the bare-metal server from the USB drive to begin the Harvester installation process. -### Begin Setup Process -You will be waiting a few minutes while the server boots from the USB drive, but you will eventually land on a page where it asks you to set up various values to use for networking and the cluster itself. -The values seen below are examples and represent how my homelab is configured. -- **Management Interface(s)**: `eno1,eno2,eno3,eno4` -- **Network Bond Mode**: `Active-Backup` -- **IP Address**: `192.168.3.254/24` *<---- **Note:** Be sure to add CIDR Notation*. -- **Gateway**: `192.168.3.1` -- **DNS Server(s)**: `1.1.1.1,1.0.0.1,8.8.8.8,8.8.4.4` -- **Cluster VIP (Virtual IP)**: `192.168.3.251` *<---- **Note**: See "VIRTUAL IP CONFIGURATION" note below.* -- **Cluster Node Token**: `19-USED-when-JOINING-more-NODES-to-EXISTING-cluster-55` -- **NTP Server(s)**: `0.suse.pool.ntp.org` - -:::caution Virtual IP Configuration -The VIP assigned to the first node in the cluster will act as a proxy to the built-in load-balancing system. It is important that you do not create a second node with the same VIP (Could cause instability in existing cluster), or use an existing VIP as the Node IP address of a new Harvester Cluster Node. -::: -:::tip -Based on your preference, it would be good to assign the device a static DHCP reservation, or use numbers counting down from **.254** (e.g. `192.168.3.254`, `192.168.3.253`, `192.168.3.252`, etc...) -::: - -### Wait for Installation to Complete -The installation process will take quite some time, but when it is finished, the Harvester Node will reboot and take you to a splash screen with the Harvester logo, with indicators as to what the VIP and Management Interface IPs are configured as, and whether or not the associated systems are operational and ready. **Be patient until both statuses say `READY`**. If after 15 minutes the status has still not changed to `READY` both for fields, see the note below. -:::caution Issues with `rancher-harvester-repo` Image -During my initial deployment efforts with Harvester v.1.1.2, I noticed that the Harvester Node never came online. That was because something bugged-out during installation and the `rancher-harvester-repo` image was not properly installed prior to node initialization. This will effectively soft-lock the node unless you reinstall the node from scratch, as the Docker Hub Registry that Harvester is looking for to finish the deployment does not exist anymore and depends on the local image bundled with the installer ISO. - -If this happens, you unfortunately need to start over and reinstall Harvester and hope that it works the second time around. No other workarounds are currently known at this time on version 1.1.2. -::: - -## Additional Harvester Nodes -If you work in a production environment, you will want more than one Harvester node to allow live-migrations, high-availability, and better load-balancing in the Harvester Cluster. The section below will outline the steps necessary to create additional Harvester nodes, join them to the existing Harvester cluster, and validate that they are functioning without issues. -### Installation Process -Not Documented Yet -### Joining Node to Existing Cluster -Not Documented Yet - -## Installing Rancher -If you plan on using Harvester for more than just running Virtual Machines (e.g. Containers), you will want to deploy Rancher inside of the Harvester Cluster in order or orchestrate the deployment, management, and rolling upgrades of various forms of Kubernetes Clusters (RKE2 Suggested). The steps below will go over the process of deploying a High-Availability Rancher environment to "adopt" Harvester as a VDI/compute platform for deploying the Kubernetes Cluster. -### Provision ControlPlane Node(s) VMs on Harvester -Not Documented Yet -### Adopt Harvester as Cluster Target -Not Documented Yet -### Deploy Production Kubernetes Cluster to Harvester -Not Documented Yet diff --git a/deployments/services/Automation Tools/Node Red.md b/deployments/services/Automation Tools/Node Red.md deleted file mode 100644 index 3bcfafa..0000000 --- a/deployments/services/Automation Tools/Node Red.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -tags: - - Node-RED - - Automation - - Docker ---- - -**Purpose**: Node-RED is a programming tool for wiring together hardware devices, APIs and online services in new and interesting ways. - -```yaml title="docker-compose.yml" -version: "3.7" - -services: - node-red: - image: nodered/node-red:latest - environment: - - TZ=America/Denver - ports: - - "1880:1880" - networks: - docker_network: - ipv4_address: 192.168.5.92 - volumes: - - /srv/containers/node-red:/data -networks: - default: - external: - name: docker_network - docker_network: - external: true - -``` - -```yaml title=".env" -Not Applicable -``` diff --git a/deployments/services/DNS/AdGuard Home.md b/deployments/services/DNS/AdGuard Home.md deleted file mode 100644 index 97a2104..0000000 --- a/deployments/services/DNS/AdGuard Home.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -tags: - - AdGuard Home - - DNS - - Docker ---- - -**Purpose**: AdGuard Home is a network-wide software for blocking ads & tracking. After you set it up, it will cover ALL your home devices, and you don’t need any client-side software for that. With the rise of Internet-Of-Things and connected devices, it becomes more and more important to be able to control your whole network. - -```yaml title="docker-compose.yml" -version: '3' - -services: - app: - image: adguard/adguardhome - ports: - - 3000:3000 - - 53:53 - - 80:80 - volumes: - - /srv/containers/adguard_home/workingdir:/opt/adguardhome/work - - /srv/containers/adguard_home/config:/opt/adguardhome/conf - restart: always - networks: - docker_network: - ipv4_address: 192.168.5.189 -networks: - default: - external: - name: docker_network - docker_network: - external: true -``` - -```jsx title=".env" -Not Applicable -``` diff --git a/deployments/services/File Services/Windows Server/DFS Namespaces with Replication.md b/deployments/services/File Services/Windows Server/DFS Namespaces with Replication.md deleted file mode 100644 index deb0189..0000000 --- a/deployments/services/File Services/Windows Server/DFS Namespaces with Replication.md +++ /dev/null @@ -1,542 +0,0 @@ ---- -tags: - - DFS - - Windows Server - - Windows - - File Services ---- - -## Purpose -If you want data available from a single, consistent UNC path while hosting it on multiple file servers, use **DFS Namespaces (DFSN)**. A namespace presents a *virtual* folder tree (for example, `\\bunny-lab.io\Projects`) whose folders point to one or more **folder targets** (actual SMB shares on your servers). -**DFS Replication (DFSR)** is a *separate* feature you configure to keep the contents of those targets in sync. - -This document walks through creating a domain-based DFS namespace and enabling DFS Replication for two servers. - -!!! info "Assumptions" - You have two Windows Server machines (e.g., `LAB-FPS-01` and `LAB-FPS-02`) running an edition that supports DFS (Standard or Datacenter), both activated, domain-joined, and using static IPs. - -### Installing Server Roles -Install the roles on **both servers**: - -* **Server Manager → Manage → Add Roles and Features** -* Click **Next** to **Server Roles** -* Expand **File and Storage Services** - * Expand **File and iSCSI Services** - * Check **File Server** - * Check **DFS Namespaces** - * Check **DFS Replication** -* **Next → Next → Install**, then finish. - -### Create & Configure Network Shares -Create (or identify) the folders you want to publish in the namespace, and share them on **each** server. Be sure to enable **Access-based Enumeration** on all of the folder shares for additional security. You only need to ensure that the files exist on one of the file servers,then you need to create empty top-level folders with the same names on the replica servers, data will be replicated automatically from the file server to the empty folders. - -Additionally, it is recommended (if possible) to set the share names to be hidden. For example `\\LAB-FPS-01\Projects$`, that way it ensures that users access the share via DFS at `\\bunny-lab.io\Projects` and users don't accidentally access the network shares directly, bypassing DFS. For example, the local path would be `Z:\Projects` but the network share would be `\\LAB-FPS-01\Projects$`. *This wouldn't break things like replication, but it would muck things up a little bit organizationally. The data would still be replicated between both servers, we just dont want users using direct server shares like that, which bypasses the high-availability and load-balancing features of DFS* - -!!! warning "What must match vs. what can differ" - - **Must exist on each server:** a shared folder to act as the *folder target* (path can differ per server). - - **Share permissions:** are **not replicated**; set them on each server. - - **NTFS permissions inside the replicated folder:** **are replicated** by DFSR and should be consistent. - - Targets do **not** have to use identical share names/paths, but keeping them consistent simplifies things. - -| **Permission Type** | **User / Group** | **Access** | Level** | -| :---- | :---- | :---- | :---- | -| Share | `Everyone` (or `Authenticated Users`) | Full Control | Best practice is to grant broad Full Control on the **share** and enforce access with NTFS. | -| NTFS | `SYSTEM` | Full Control | Required for DFSR service. | -| NTFS | `Share_Admins` | Full Control | Optional admin group for data management. | -| NTFS | *Business groups needing access* | Modify | Grant least privilege to required users/groups. | - -!!! info "Note On Inheritance" - Disabling inheritance is **not required** for DFS/DFSR. Keep it enabled unless you have a clear reason to flatten ACLs; inheritance often reduces long-term admin overhead. - -### DFS Breakdown -A **namespace** is a logical view like `\\bunny-lab.io\Projects`. Inside it, you create DFS **folders** (e.g., `Scripting`) that point to one or more **folder targets**, such as: - -* `\\LAB-FPS-01\Projects$\Scripting` -* `\\LAB-FPS-02\Projects$\Scripting` - -The namespace root itself isn't where you store data; it's a directory of links. Place data in the folder targets the DFS folder points to. - -### DFS Configuration -You can run these steps from either server (or any admin workstation with the RSAT tools). DFSN configuration is stored in AD and on namespace servers and applies across members automatically. - -#### Create Namespace - -* **Server Manager → Tools → DFS Management** -* Right-click **Namespaces** → **New Namespace...** - * Choose a server to host the namespace (e.g., `LAB-FPS-01`) → **Next** - * Name the namespace (e.g., `Projects`) → **Next** - * You can leave **Edit Settings** at defaults; those control the local folder that backs the namespace root, not your data. - * Choose **Domain-based namespace** and check **Enable Windows Server 2008 mode** (required for larger scale and Access-based enumeration). - * Resulting path: `\\bunny-lab.io\Projects` - * **Next → Create** - -#### Make Namespace Highly-Available -We have to perform an extra step to ensure that every file server can act as within a multi-master context, allowing for high availability. To do this in this example, we will add `LAB-FPS-02` as a secondary namespace server for every namespace that we create. - -- Right-Click **DFS Management** > **Namespaces** > `\\bunny-lab.io\Projects` -- Click **Add Namespace Server...** -- Under "Namespace Server" enter `LAB-FPS-02` then click **OK**. - -#### Enable Access-Based Enumeration on Namespace - -- Right-Click **DFS Management** > **Namespaces** > `\\bunny-lab.io\Projects` -- Click **Properties** -- Click **Advanced** -- Check **Enable access-based enumeration for this namespace** -- Click **OK** - -#### Link Folders to Namespace -Create the DFS folders and add folder targets: - -* Right-click the new namespace (e.g., `\\bunny-lab.io\Projects`) → **New Folder...** - * **Name:** `Scripting` - * **Add** folder targets (one per server), e.g.: - * `\\LAB-FPS-01\Projects$\Scripting` - * `\\LAB-FPS-02\Projects$\Scripting` - * You can simply copy-paste the previous server location and substitute the hostname (e.g. switching `01` to `02`) instead of browsing for the folder. - * You *may* be prompted to create the folder because it does not exist on `LAB-FPS-02`, in this circumstance, you can tell it to create the folder automatically with read-only permissions. *Don't worry, when replication from `LAB-FPS-01` occurs, NTFS permissions will be overwritten to the correct users and groups.* - * When prompted *"Create a replication group to synchronize the folder targets?"*, click **Yes** to launch the DFS Replication wizard. - -!!! info "**Be patient**" - The Replication wizard can take ~1 minute to appear. - -#### Configure Replication Group -In the Replication wizard that appears after about a minute, you can configure the replication group for the folder: - -!!! bug "If Wizard did Not Appear (or Crashed)" - In my homelab testing, I had two times when the wizard crashed or simply never opened. If this happens to you, you can manually re-trigger the wizard for the target folder by right-clicking the folder (e.g. `\\bunny-lab.io\Projects\Scripting`) and selecting **Replicate Folder**. - -* **Replication Group Name**: *(leave as suggested)* -* **Replicated Folder Name**: *(leave as suggested)* -* **Next → Next** -* **Primary member**: pick the server with the **most up-to-date** copy of the data (e.g., `LAB-FPS-01`). - -!!! abstract "Replication Behavior and Expectations" - When you first create a replication group, DFSR needs a baseline copy of the data to start from. You designate one server as the Primary Member to serve as that baseline. (e.g. `LAB-FPS-01`) During the first sync, DFSR assumes that whatever exists on the primary member's folder is the "truth." So if the same file exists on another server (e.g. `LAB-FPS-02`) but with different timestamps, sizes, or hashes, the primary member's copy wins - but only during this first synchronization. After that initial sync is complete, the "primary" flag loses all authority. Replication becomes multi-master, meaning every member can make changes, and DFSR uses its conflict resolution algorithm (based on version vectors, update sequence numbers, and timestamps) to decide which change wins going forward. In other words, no server remains “the boss” after initialization. Files unique to other member servers that only exist on them will not be wiped and will be replicated across all member servers including the primary member. - -* **Topology**: `Full mesh` (good for two servers; for many sites, consider hub-and-spoke). -* **Replication schedule**: leave **Full** (24x7) unless you need bandwidth windows. -* **Create** - -!!! success "Replication group created" - You should see green ticks for the following. Give everything some time to replicate as it depends on active directory replication speeds to push out the configuration across the DFS member servers and begin the replication. - - - ✅Create replication group - - ✅Create members - - ✅Update folder security - - ✅Create replicated folder - - ✅Create membership objects - - ✅Update folder properties - - ✅Create connections - -### Troubleshooting / Diagnostics -#### Checking DFS Status -You may want to put together a simple table report of the DFS namespaces, replication info, and target folders. You can run the following powershell script to generate a nice table-based report of the current structure of the DFS namespaces in your domain. - -??? example "Powershell Reporting Script" - ```powershell - # Automatically detect current AD domain and use it as DFS prefix - try { - $Domain = ([System.DirectoryServices.ActiveDirectory.Domain]::GetCurrentDomain()).Name - $DomainPrefix = "\\$Domain" - } catch { - Write-Warning "Unable to detect domain automatically. Falling back to manual value." - $DomainPrefix = "\\bunny-lab.io" - } - - Import-Module DFSN -ErrorAction Stop - Import-Module DFSR -ErrorAction Stop - - function Get-ServerNameFromPath { - param([string]$Path) - if ([string]::IsNullOrWhiteSpace($Path)) { return $null } - if ($Path -like "\\*") { return ($Path -split '\\')[2] } - return $null - } - function Get-Max3 { - param([int[]]$Values) - if (-not $Values) { return 0 } - return (($Values | Measure-Object -Maximum).Maximum) - } - - # Build: GroupName (lower) -> memberships[] - $allGroups = Get-DfsReplicationGroup -ErrorAction SilentlyContinue - $groupMembershipMap = @{} - foreach ($g in $allGroups) { - $ms = Get-DfsrMembership -GroupName $g.GroupName -ErrorAction SilentlyContinue - $groupMembershipMap[$g.GroupName.ToLower()] = $ms - } - - # Flatten all memberships for regex fallback - $allMemberships = @() - foreach ($arr in $groupMembershipMap.Values) { if ($arr) { $allMemberships += $arr } } - - $rows = New-Object System.Collections.Generic.List[psobject] - - # Enumerate namespace roots - $roots = Get-DfsnRoot -ErrorAction Stop | Where-Object { $_.Path -like "$DomainPrefix\*" } - - Write-Host "DFS Namespace and Replication Overview" -ForegroundColor Cyan - Write-Host "------------------------------------------------------`n" - - foreach ($root in $roots) { - - $rootPath = $root.Path - $rootLeaf = ($rootPath -split '\\')[-1] - - $nsServers = @() - $rootTargets = Get-DfsnRootTarget -Path $rootPath -ErrorAction SilentlyContinue - foreach ($rt in $rootTargets) { - $srv = Get-ServerNameFromPath $rt.TargetPath - if ($srv) { $nsServers += $srv } - } - - # Folders under this root - $folders = Get-DfsnFolder -Path "$rootPath\*" -ErrorAction SilentlyContinue | Sort-Object Path - - foreach ($f in $folders) { - $namespaceFull = $f.Path - $leaf = ($f.Path -split '\\')[-1] - - # DFSN folder targets - $targets = Get-DfsnFolderTarget -Path $f.Path -ErrorAction SilentlyContinue - $targets = @($targets | Sort-Object { Get-ServerNameFromPath $_.TargetPath }) # ensure array - - # Map to DFSR group by naming; fallback to regex on ContentPath - $candidateGroup = ((($rootPath -replace '^\\\\','') + '\' + $leaf).ToLower()) - if ($groupMembershipMap.ContainsKey($candidateGroup)) { - $msForFolder = $groupMembershipMap[$candidateGroup] - } else { - $escapedRootLeaf = [regex]::Escape($rootLeaf) - $escapedLeaf = [regex]::Escape($leaf) - $regex = "\\$escapedRootLeaf\\$escapedLeaf($|\\)" - $msForFolder = $allMemberships | Where-Object { $_.ContentPath -imatch $regex } - } - $msForFolder = @($msForFolder) # normalize to array - - # Build aligned rows: one per target - $targetLines = @() - $replLines = @() - - foreach ($t in $targets) { - $tServer = Get-ServerNameFromPath $t.TargetPath - $targetLines += $t.TargetPath - - $msForServer = $null - if ($msForFolder.Count -gt 0) { - $msForServer = $msForFolder | Where-Object { $_.ComputerName -ieq $tServer } | Select-Object -First 1 - } - if ($msForServer -and $msForServer.ContentPath) { $replLines += $msForServer.ContentPath } else { $replLines += '' } - } - - # Max line count for row expansion (PS 5.1 safe) - $maxLines = Get-Max3 @($targetLines.Count, $replLines.Count, $nsServers.Count) - - for ($i = 0; $i -lt $maxLines; $i++) { - - # Precompute values (PS 5.1: no inline-if in hashtables) - $nsVal = '' - if ($i -eq 0) { $nsVal = $namespaceFull } - - $targetVal = '' - if ($i -lt $targetLines.Count) { $targetVal = $targetLines[$i] } - - $replVal = '' - if ($i -lt $replLines.Count) { $replVal = $replLines[$i] } - - $nsServerVal = '' - if ($i -lt $nsServers.Count) { $nsServerVal = $nsServers[$i] } - - $row = [PSCustomObject]@{ - 'Namespace' = $nsVal - 'Member Folder Target(s)' = $targetVal - 'Replication Locations' = $replVal - 'Namespace Servers' = $nsServerVal - } - $rows.Add($row) | Out-Null - } - } - } - - # Render as a PowerShell bordered grid with one-space left/right padding in every cell - function Write-DfsGrid { - [CmdletBinding()] - param( - [Parameter(Mandatory)] - [System.Collections.IEnumerable]$Data, - - [string[]]$Columns = @('Namespace','Member Folder Target(s)','Replication Locations','Namespace Servers'), - - # Reasonable max widths; tune to your console (these are content+padding widths) - [int[]]$MaxWidths = @(70, 70, 52, 30), - - [switch]$Ascii # use +-| instead of box-drawing if your console garbles Unicode - ) - - # Ensure arrays align - if ($MaxWidths.Count -lt $Columns.Count) { - $pad = New-Object System.Collections.Generic.List[int] - $pad.AddRange($MaxWidths) - for ($i=$MaxWidths.Count; $i -lt $Columns.Count; $i++) { $pad.Add(40) } - $MaxWidths = $pad.ToArray() - } - - # Characters - if ($Ascii) { - $H = @{ tl='+'; tr='+'; bl='+'; br='+'; hz='-'; vt='|'; tj='+'; mj='+'; bj='+' } - } else { - # Box-drawing - $H = @{ tl='┌'; tr='┐'; bl='└'; br='┘'; hz='─'; vt='│'; tj='┬'; mj='┼'; bj='┴' } - try { [Console]::OutputEncoding = [Text.UTF8Encoding]::UTF8 } catch {} - } - - function TruncPad([string]$s, [int]$w) { - if ($null -eq $s) { $s = '' } - $s = $s -replace '\r','' -replace '\t',' ' - if ($s.Length -le $w) { return $s.PadRight($w, ' ') } - if ($w -le 1) { return $s.Substring(0, $w) } - return ($s.Substring(0, $w-1) + '…') - } - - # Materialize and compute widths (include one-space left/right padding for header and data) - $rows = @($Data | ForEach-Object { - $o = @{} - foreach ($c in $Columns) { $o[$c] = [string]($_.$c) } - [pscustomobject]$o - }) - - $widths = @() - for ($i=0; $i -lt $Columns.Count; $i++) { - $col = $Columns[$i] - # Start with header length including padding - $max = (" " + $col + " ").Length - foreach ($r in $rows) { - $len = (" " + [string]$r.$col + " ").Length - if ($len -gt $max) { $max = $len } - } - $widths += [Math]::Min($max, $MaxWidths[$i]) - } - - # Line builders - function DrawTop() { - $line = $H.tl - for ($i = 0; $i -lt $widths.Count; $i++) { - $line += ($H.hz * $widths[$i]) - if ($i -lt ($widths.Count - 1)) { - $line += $H.tj - } else { - $line += $H.tr - } - } - $line - } - function DrawMid([string[]]$Columns, [int[]]$widths, $H) { - $line = $H.vt - for ($i=0; $i -lt $widths.Count; $i++) { - $line += TruncPad (" " + $Columns[$i] + " ") $widths[$i] - $line += $H.vt - } - $line - } - function DrawSep() { - $line = $H.vt - for ($i=0; $i -lt $widths.Count; $i++) { - $line += ($H.hz * $widths[$i]) - $line += $H.vt - } - $line - } - function DrawHeaderSep() { - $line = $H.vt - for ($i=0; $i -lt $widths.Count; $i++) { - $line += ($H.hz * $widths[$i]) - $line += $H.vt - } - $line - } - function DrawBottom() { - $line = $H.bl - for ($i = 0; $i -lt $widths.Count; $i++) { - $line += ($H.hz * $widths[$i]) - if ($i -lt ($widths.Count - 1)) { - $line += $H.bj - } else { - $line += $H.br - } - } - $line - } - function DrawRow($r, [string[]]$Columns, [int[]]$widths, $H) { - $line = $H.vt - for ($i=0; $i -lt $widths.Count; $i++) { - $val = [string]$r.($Columns[$i]) - $line += TruncPad (" " + $val + " ") $widths[$i] - $line += $H.vt - } - $line - } - - # Render with group separators between namespaces (when the Namespace cell is non-empty) - Write-Host (DrawTop) - Write-Host (DrawMid -Columns $Columns -widths $widths -H $H) - Write-Host (DrawHeaderSep) - - $first = $true - foreach ($r in $rows) { - if (-not $first -and ([string]$r.$($Columns[0])) ) { - # Namespace changed → draw a separator - Write-Host (DrawSep) - } - $first = $false - Write-Host (DrawRow -r $r -Columns $Columns -widths $widths -H $H) - } - - Write-Host (DrawBottom) - } - - Write-DfsGrid -Data $rows - ``` - -#### Fixing Inconsistent DFS Management GUI -Sometimes the GUI for managing DFS becomes "inconsistent" whereas the namespaces and replication groups are different between member servers, and may be missing namspaces or missing replication groups. DFS Management is an MMC snap-in. MMC persists per-user console state under `%APPDATA%\Microsoft\MMC\`. If that state gets out of sync (common after service hiccups or server crashes), the snap-in can render partial/incorrect namespace/replication trees even when DFS itself is fine. Deleting the cached dfsmgmt* console forces a fresh enumeration. We will also include a few extra commands for extra thouroughness. - -Before anything, we want to make sure that active directory itself is not having replication issues, as this would be a deeper, more complicated issue. Run the following command on one of your domain controllers: -```powershell -repadmin /syncall /AdeP -repadmin /replsummary -``` - -If AD-level replication is successful and timely, you can proceed to run the commands below (one-line-at-a-time): -```sh -# Pull-Down DFS Configuration from Active Directory & Restart DFS -dfsrdiag pollad -net stop dfsr -net start dfsr - -# Clear DFS Management Snap-In Cache -taskkill /im mmc.exe /f -del "%appdata%\Microsoft\MMC\dfsmgmt*" -dfsmgmt.msc -``` - -!!! success "DFS Management GUI Restored" - At this point, the DFS Management snap-in (should) be successfully showing all of the DFS namespaces and replication groups when you re-open "DFS Management". - -#### Check Replication Progress -You may want to check that replication is occurring bi-directionally between every member server in your DFS deployment. I wrote a script below that effectively shows you every replication group and each directional backlog status. - -```powershell -# --- CONFIG --- -$Members = @("LAB-FPS-01","LAB-FPS-02") -$SummarizeAcrossFolders = $true # $true = one line per direction per RG; $false = per-folder lines - -function Invoke-DfsrBacklogStatus { - param( - [Parameter(Mandatory)] [string] $RG, - [Parameter(Mandatory)] [string] $RF, - [Parameter(Mandatory)] [string] $Send, - [Parameter(Mandatory)] [string] $Recv - ) - - $out = & dfsrdiag backlog /rgname:"$RG" /rfname:"$RF" /sendingmember:"$Send" /receivingmember:"$Recv" 2>&1 | Out-String - $outTrim = ($out -split "`r?`n" | ForEach-Object { $_.Trim() }) | Where-Object { $_ -ne "" } - - if ($out -match 'No Backlog') { - return [pscustomobject]@{ Status="No Backlog"; Count=0; Detail=$null } - } - - $count = $null - $countLine = $outTrim | Where-Object { $_ -match '(?i)backlog' } | Select-Object -First 1 - if ($countLine -and ($countLine -match '(\d+)')) { $count = [int]$matches[1] } - - $detail = ($outTrim | Select-Object -First 8) -join " | " - - return [pscustomobject]@{ - Status = if ($count -ne $null) { "Backlog: $count" } else { "Backlog/Check Output" } - Count = $count - Detail = $detail - } -} - -$groups = Get-DfsReplicationGroup | Sort-Object GroupName - -foreach ($g in $groups) { - $rg = $g.GroupName - $rfs = Get-DfsReplicatedFolder -GroupName $rg | Sort-Object FolderName - - Write-Host "" - Write-Host ("== Replication Group: {0} ==" -f $rg) - - foreach ($send in $Members) { - foreach ($recv in $Members) { - if ($send -eq $recv) { continue } - - if ($SummarizeAcrossFolders) { - $worstCount = 0 - $nonZero = @() - $errorsOrDetails = @() - - foreach ($rfObj in $rfs) { - $rf = $rfObj.FolderName - $res = Invoke-DfsrBacklogStatus -RG $rg -RF $rf -Send $send -Recv $recv - - if ($res.Status -ne "No Backlog") { - $nonZero += [pscustomobject]@{ RF=$rf; Status=$res.Status; Count=$res.Count; Detail=$res.Detail } - if ($res.Count -ne $null -and $res.Count -gt $worstCount) { $worstCount = $res.Count } - - # ✅ FIX: ${rf} avoids the ':' parsing issue - if ($res.Detail) { $errorsOrDetails += "RF=${rf}: $($res.Detail)" } - } - } - - if ($nonZero.Count -eq 0) { - Write-Host ("{0} -> {1}: No Backlog" -f $send, $recv) - } else { - if ($worstCount -gt 0) { - Write-Host ("{0} -> {1}: Backlog (max {2} across RFs)" -f $send, $recv, $worstCount) - } else { - Write-Host ("{0} -> {1}: Backlog/Errors (see details)" -f $send, $recv) - } - - $errorsOrDetails | Select-Object -First 5 | ForEach-Object { Write-Host (" - {0}" -f $_) } - if ($errorsOrDetails.Count -gt 5) { Write-Host " - ... (more omitted)" } - } - } - else { - foreach ($rfObj in $rfs) { - $rf = $rfObj.FolderName - $res = Invoke-DfsrBacklogStatus -RG $rg -RF $rf -Send $send -Recv $recv - - if ($res.Status -eq "No Backlog") { - Write-Host ("{0} -> {1} [{2}]: No Backlog" -f $send, $recv, $rf) - } else { - Write-Host ("{0} -> {1} [{2}]: {3}" -f $send, $recv, $rf, $res.Status) - if ($res.Detail) { Write-Host (" - {0}" -f $res.Detail) } - } - } - } - } - } -} -``` - -!!! example "Example Output" - You will see output like the following when you run the script. - - ```powershell - == Replication Group: bunny-lab.io\music\fl studio plugins == - LAB-FPS-01 -> LAB-FPS-02: No Backlog - LAB-FPS-02 -> LAB-FPS-01: No Backlog - - == Replication Group: bunny-lab.io\music\personal music == - LAB-FPS-01 -> LAB-FPS-02: No Backlog - LAB-FPS-02 -> LAB-FPS-01: No Backlog - - == Replication Group: bunny-lab.io\music\shared music == - LAB-FPS-01 -> LAB-FPS-02: No Backlog - LAB-FPS-02 -> LAB-FPS-01: No Backlog - - == Replication Group: bunny-lab.io\projects\coding == - LAB-FPS-01 -> LAB-FPS-02: No Backlog - LAB-FPS-02 -> LAB-FPS-01: No Backlog - ``` \ No newline at end of file diff --git a/deployments/services/Security and Utility/Cyberchef.md b/deployments/services/Security and Utility/Cyberchef.md deleted file mode 100644 index 0a91dff..0000000 --- a/deployments/services/Security and Utility/Cyberchef.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -tags: - - CyberChef - - Security - - Docker ---- - -**Purpose**: The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis. - -```yaml title="docker-compose.yml" -version: "3.8" -services: - app: - image: mpepping/cyberchef:latest - container_name: cyberchef - environment: - - TZ=America/Denver - ports: - - 8000:8000 - restart: always - networks: - docker_network: - ipv4_address: 192.168.5.55 - -networks: - default: - external: - name: docker_network - docker_network: - external: true -``` - -```jsx title=".env" -N/A -``` diff --git a/deployments/services/Security and Utility/IT Tools.md b/deployments/services/Security and Utility/IT Tools.md deleted file mode 100644 index a48acdf..0000000 --- a/deployments/services/Security and Utility/IT Tools.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -tags: - - IT-Tools - - Security - - Docker ---- - -**Purpose**: Collection of handy online tools for developers, with great UX. - -```yaml title="docker-compose.yml" -version: "3" - -services: - server: - image: corentinth/it-tools:latest - container_name: it-tools - environment: - - TZ=America/Denver - restart: always - ports: - - "80:80" - networks: - docker_network: - ipv4_address: 192.168.5.16 - -networks: - docker_network: - external: true -``` - -```yaml title=".env" -Not Applicable -``` diff --git a/deployments/services/authentication/keycloak/oauth2/Gitea OAuth2.md b/deployments/services/authentication/keycloak/oauth2/Gitea OAuth2.md deleted file mode 100644 index f35dc12..0000000 --- a/deployments/services/authentication/keycloak/oauth2/Gitea OAuth2.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -tags: - - Gitea - - Keycloak - - OAuth2 - - Authentication ---- - -### OAuth2 Configuration -These are variables referenced by the associated service to connect its authentication system to [Keycloak](../deployment.md). - -| **Parameter** | **Value** | -| :--- | :--- | -| Authentication Name | `auth-bunny-lab-io` | -| OAuth2 Provider | `OpenID Connect` | -| Client ID (Key) | `git-bunny-lab-io` | -| Client Secret | `https://auth.bunny-lab.io > Clients > git-bunny-lab-io > Credentials > Client Secret` | -| OpenID Connect Auto Discovery URL | `https://auth.bunny-lab.io/realms/master/.well-known/openid-configuration` | -| Skip Local 2FA | Yes | - diff --git a/deployments/services/authentication/keycloak/oauth2/deployment.md b/deployments/services/authentication/keycloak/oauth2/deployment.md deleted file mode 100644 index fefa072..0000000 --- a/deployments/services/authentication/keycloak/oauth2/deployment.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -tags: - - Keycloak - - OAuth2 - - Authentication - - Docker ---- - -You can deploy Keycloak via a [docker-compose stack](../deployment.md) found within the "Containerization" section of the documentation. - diff --git a/deployments/services/cPanel/Creating Email Server.md b/deployments/services/cPanel/Creating Email Server.md deleted file mode 100644 index afef9f1..0000000 --- a/deployments/services/cPanel/Creating Email Server.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -tags: - - cPanel - - Email ---- - -## Purpose -This documentation helps you deploy an email server within a cPanel hosted environment. - -!!! note "Assumptions" - It is assumed that the cPanel environment is set up (prior) to following this documentation, as deploying cPanel itself is not covered in this document. - -## Step - -### Sub-Step - diff --git a/deployments/services/email/Microsoft Exchange/Configuring ACME Letsencrypt Bot.md b/deployments/services/email/Microsoft Exchange/Configuring ACME Letsencrypt Bot.md deleted file mode 100644 index 3a513e8..0000000 --- a/deployments/services/email/Microsoft Exchange/Configuring ACME Letsencrypt Bot.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -tags: - - Microsoft Exchange - - Lets Encrypt - - Email ---- - -**Purpose**: If you want to set up automatic Let's Encrypt SSL certificates on a Microsoft Exchange server, you have to go through a few steps to install the WinACME bot, and configure it to automatically renew certificates. - -!!! note "ACME Bot Provisioning Considerations" - This document assumes you want a fully-automated one-liner command for configuring the ACME Bot, it is also completely valid to go step-by-step through the bot to configure the SSL certificate, the IIS server, etc, and it will automatically create a Scheduled Task to renew on its own. The whole process is very straight-forward with most answers being the default option. - -### Download the Win-ACME Bot: - -* Log into the on-premise Exchange Server via Datto RMM -* Navigate to: [https://www.win-acme.com/](https://www.win-acme.com/) - * On the top-right of the website, you will see a "**Download**" button with the most recent version of the Win-ACME bot -* Extract the contents of the ZIP file to "**C:\\Program Files (x86)\\Lets Encrypt**" - * Make the "**Lets Encrypt**" folder if it does not already exist - -### Configure `settings_default.json`: - -* The next step involves us making a modification to the configuration of the Win-ACME bot that allows us to export the necessary private key data for Exchange -* Using a text editor, open the "**settings\_default.json**" file - * Look for the setting called "**PrivateKeyExportable**" and change the value from "**false**" to "**true**" - * Save and close the file - -### Download and Install the SSL Certificate: - -* Open an administrative Command Line (DO NOT USE POWERSHELL) -* Navigate to the Let's Encrypt bot directory: `CD "C:\Program Files (x86)\Lets Encrypt"` -* Invoke the bot to automatically download and install the certificate into the IIS Server that Exchange uses to host the Exchange Server - * Be sure to change the placeholder subdomains to match the domain of the actual Exchange Server - * (e.g. "**mail.example.org**" | "**autodiscover.example.org**") - ``` - wacs.exe --target manual --host mail.example.org,autodiscover.example.org --certificatestore My --acl-fullcontrol "network service,administrators" --installation iis,script --installationsiteid 1 --script "./Scripts/ImportExchange.ps1" --scriptparameters "'{CertThumbprint}' 'IIS,SMTP,IMAP' 1 '{CacheFile}' '{CachePassword}' '{CertFriendlyName}'" --verbose - ``` - -* When the command is running, it will ask for an email address for alerts and abuse notifications, just put "**infrastructure@bunny-lab.io**" -* If you run into any unexpected errors that result in anything other than exiting with a status "0", consult with Nicole Rappe to proceed - * Check that the domain of the Exchange Server is reachable on port 80 as Let's Encrypt uses this to build the cert. - * Searching the external IP of the server on [Shodan](https://www.shodan.io/) will reveal all open ports. - -### Troubleshooting: -If you find that any of the services such as [https://mail.example.org/ecp](https://mail.example.org/ecp), [https://autodiscover.example.org](https://autodiscover.example.org), or [https://mail.example.org/owa](https://mail.example.org/owa) do not let you log in, proceed with the steps below to correct the "Certificate Binding" in IIS Manager: - -* Open "**Server Manager**" > Tools > "**Internet Information Services (IIS) Manager**" -* Expand the "**Connections**" server tree on the left-hand side of the IIS Manager -* Expand the "**Sites**" folder - * Click on "**Default Web Site**" - * On the right-hand Actions menu, click on "**Bindings...**" - * A table will appear with different endpoints on the Exchange server > What you are looking for is an entry that looks like the following: - * **Type**: https - * **Host Name**: autodiscover.example.org - * **Port**: 443 - * Double-click on the row, or click one then click the "**Edit**" button to open the settings for that endpoint - * Under "**SSL Certificate**" > Make sure the certificate name matches the following format: "**\[Manual\] autodiscover.example.org @ YYYY/MM/DD**" - * If it does not match the above, use the dropdown menu to correct it and click the "**OK**" button - * **Type**: https - * **Host Name**: mail.example.org - * **Port**: 443 - * Repeat the steps seen above, except this time for "**mail.example.org**" - * Click on "**Exchange Back End**" - * On the right-hand Actions menu, click on "**Bindings...**" - * A table will appear with different endpoints on the Exchange server > What you are looking for is an entry that looks like the following: - * **Type**: https - * **Host Name**: - * **Port**: 444 - * Repeat the steps seen above, ensuring that the "**\[Manual\] autodiscover.example.org @ YYYY/MM/DD**" certificate is selected and applied - * Click the "**OK**" button - * On the left-hand menu under "**Connections**" in IIS Manager, click on the server name itself - * (e.g. "**EXAMPLE-EXCHANGE (DOMAIN\\dptadmin**") - * On the right-hand "**Actions**" menu > Under "Manage Server" > Select "Restart" - * Wait for the IIS server to restart itself, then try accessing the webpages for Exchange that were exhibiting issues logging in - -### Additional Documentation: - -* [https://www.alitajran.com/install-free-lets-encrypt-certificate-in-exchange-server/](https://www.alitajran.com/install-free-lets-encrypt-certificate-in-exchange-server/) \ No newline at end of file diff --git a/deployments/services/email/iRedMail/Quick Server Settings.md b/deployments/services/email/iRedMail/Quick Server Settings.md deleted file mode 100644 index d3f183a..0000000 --- a/deployments/services/email/iRedMail/Quick Server Settings.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -tags: - - IredMail - - Email ---- - -| Server | Port(s) | Security | Auth Method | Username | -|:------------------|:----------------------------------------------|:----------|:----------------|:-------------------| -| `mail.bunny-lab.io` | **IMAP:** 143 `Internal`, 993 `External`
**SMTP:** 587, 25 `Fallback` | STARTTLS | Normal Password | user@bunny-lab.io | diff --git a/deployments/services/index.md b/deployments/services/index.md deleted file mode 100644 index 36ed0f2..0000000 --- a/deployments/services/index.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -tags: - - Services - - Index - - Documentation ---- - -# Services -## Purpose -Deployable services and applications in the lab (auth, email, monitoring, etc). - -## Includes -- Service deployments and configs -- Dependencies and integrations -- Operational notes specific to the service - -## New Document Template -````markdown -# -## Purpose - - -!!! info "Assumptions" - - - - - -## Dependencies -- - -## Procedure -```sh -# Commands or deployment steps -``` - -## Validation -- - -## Rollback -- -```` diff --git a/deployments/services/productivity/OnlyOffice EE.md b/deployments/services/productivity/OnlyOffice EE.md deleted file mode 100644 index 4819d6c..0000000 --- a/deployments/services/productivity/OnlyOffice EE.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -tags: - - OnlyOffice - - Productivity - - Docker ---- - -**Purpose**: ONLYOFFICE offers a secure online office suite highly compatible with MS Office formats. Generally used with Nextcloud to edit documents directly within the web browser. - -```yaml title="docker-compose.yml" -version: '3' - -services: - app: - image: onlyoffice/documentserver-ee - ports: - - 80:80 - - 443:443 - volumes: - - /srv/containers/onlyoffice/DocumentServer/logs:/var/log/onlyoffice - - /srv/containers/onlyoffice/DocumentServer/data:/var/www/onlyoffice/Data - - /srv/containers/onlyoffice/DocumentServer/lib:/var/lib/onlyoffice - - /srv/containers/onlyoffice/DocumentServer/db:/var/lib/postgresql - - /srv/containers/onlyoffice/DocumentServer/fonts:/usr/share/fonts/truetype/custom - - /srv/containers/onlyoffice/DocumentServer/forgotten:/var/lib/onlyoffice/documentserver/App_Data/cache/files/forgotten - - /srv/containers/onlyoffice/DocumentServer/rabbitmq:/var/lib/rabbitmq - - /srv/containers/onlyoffice/DocumentServer/redis:/var/lib/redis - labels: - - "traefik.enable=true" - - "traefik.http.routers.cyberstrawberry-onlyoffice.rule=Host(`office.cyberstrawberry.net`)" - - "traefik.http.routers.cyberstrawberry-onlyoffice.entrypoints=websecure" - - "traefik.http.routers.cyberstrawberry-onlyoffice.tls.certresolver=myresolver" - - "traefik.http.services.cyberstrawberry-onlyoffice.loadbalancer.server.port=80" - - "traefik.http.routers.cyberstrawberry-onlyoffice.middlewares=onlyoffice-headers" - - "traefik.http.middlewares.onlyoffice-headers.headers.customrequestheaders.X-Forwarded-Proto=https" - #- "traefik.http.middlewares.onlyoffice-headers.headers.accessControlAllowOrigin=*" - environment: - - JWT_ENABLED=true - - JWT_SECRET=REDACTED #SET THIS TO SOMETHING SECURE - restart: always - networks: - docker_network: - ipv4_address: 192.168.5.143 -networks: - default: - external: - name: docker_network - docker_network: - external: true -``` - -```yaml title=".env" -Not Applicable -``` -:::tip -If you wish to use this in a non-commercial homelab environment without limits, [this script](https://wiki.muwahhid.ru/ru/Unraid/Docker/Onlyoffice-Document-Server) does an endless trial without functionality limits. -``` -docker stop office-document-server-ee -docker rm office-document-server-ee -rm -r /mnt/user/appdata/onlyoffice/DocumentServer -sleep 5 - -``` - -Docker Run Method: -``` -docker run -d --name='office-document-server-ee' --net='bridge' -e TZ="Europe/Moscow" -e HOST_OS="Unraid" -e 'JWT_ENABLED'='true' -e 'JWT_SECRET'='mySecret' -p '8082:80/tcp' -p '4432:443/tcp' -v '/mnt/user/appdata/onlyoffice/DocumentServer/logs':'/var/log/onlyoffice':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/data':'/var/www/onlyoffice/Data':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/lib':'/var/lib/onlyoffice':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/db':'/var/lib/postgresql':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/fonts':'/usr/share/fonts/truetype/custom':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/forgotten':'/var/lib/onlyoffice/documentserver/App_Data/cache/files/forgotten':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/rabbitmq':'/var/lib/rabbitmq':'rw' -v '/mnt/user/appdata/onlyoffice/DocumentServer/redis':'/var/lib/redis':'rw' 'onlyoffice/documentserver-ee' -``` -::: - diff --git a/reference/Applications/Email/iRedMail Connection Settings.md b/reference/Applications/Email/iRedMail Connection Settings.md new file mode 100644 index 0000000..756a321 --- /dev/null +++ b/reference/Applications/Email/iRedMail Connection Settings.md @@ -0,0 +1,18 @@ +--- +tags: + - IredMail + - Email +--- + +## Purpose +Use this reference for iredmail connection settings and the environment-specific values recorded below. + +!!! info "Recorded Connection Settings" + These values belong to the iRedMail example. Use the email guide to select the matching mail-server implementation before configuring a client. + +| Server | Port(s) | Security | Auth Method | Username | +|:------------------|:----------------------------------------------|:----------|:----------------|:-------------------| +| `mail.bunny-lab.io` | **IMAP:** 143 `Internal`, 993 `External`
**SMTP:** 587, 25 `Fallback` | STARTTLS | Normal Password | user@bunny-lab.io | + +## Related Documentation +- [Related Email Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/Applications/Email/index.md b/reference/Applications/Email/index.md new file mode 100644 index 0000000..00fd9c0 --- /dev/null +++ b/reference/Applications/Email/index.md @@ -0,0 +1,31 @@ +--- +tags: + - Email + - Mailcow + - Microsoft Exchange +--- + +# Email +## Purpose +Choose the email environment before following a deployment or repair procedure. Mailcow with PMG, iRedMail, Microsoft Exchange, and the cPanel scaffold describe separate configurations; their presence here does not establish that they are all active in the lab. + +## Includes +- Mailcow and PMG integration +- iRedMail deployment and connection settings +- Exchange maintenance and DAG recovery + +## Mailcow and PMG +- [Deploy Mailcow](<../../../deployments/Applications/Email/mailcow.md>) — Configure the mail service and its documented proxy arrangement. +- [Integrate PMG with Mailcow](<../../../deployments/Applications/Email/Proxmox Mail Gateway/Integrate PMG with Mailcow.md>) — Follow the gateway, DNS, NAT, and mail-flow integration. +- [Repair Trusted Mail Delivery](<../../../workflows/Applications/Email/Proxmox Mail Gateway/Repair Trusted Mail Delivery Between PMG and Mailcow.md>) — Investigate sender-validation or relay-trust failures after integration. + +## iRedMail +- [Deploy iRedMail](<../../../deployments/Applications/Email/iRedMail/Deploy iRedMail.md>) — Follow the separately documented mail-server implementation. +- [Client Connection Settings]() — Find the recorded protocol and server settings. +- [Inspect the SMTP Queue](<../../../workflows/Applications/Email/iRedMail/Inspect the Outgoing SMTP Queue.md>) — Investigate outgoing messages on the iRedMail environment. + +## Microsoft Exchange +- [Exchange SE Rolling Updates](<../../../workflows/Applications/Email/Microsoft Exchange/Perform Exchange SE DAG Rolling Updates.md>) — Use the complete maintenance cycle for the explicitly described three-member example. +- [DAG Database Copy Repairs](<../../../workflows/Applications/Email/Microsoft Exchange/Manage DAG Database Copies.md>) — Review the older example and version context before applying its recovery commands. +- [Cumulative Update Preparation Notes](<../../../workflows/Applications/Email/Microsoft Exchange/Prepare for Cumulative Updates.md>) — Review the separate preparation procedure and its environment assumptions. +- [Certificate Packaging](<../../../workflows/Identity and Certificates/Certificates/Convert Certificates to PFX.md>) — Find the documented certificate conversion workflow. diff --git a/reference/Applications/Files and Collaboration/Rclone Command Reference.md b/reference/Applications/Files and Collaboration/Rclone Command Reference.md new file mode 100644 index 0000000..a74e547 --- /dev/null +++ b/reference/Applications/Files and Collaboration/Rclone Command Reference.md @@ -0,0 +1,101 @@ +--- +tags: + - Rclone + - PowerShell + - Synchronization + - Google Drive +--- + +## Purpose +This document explains the practical differences between the rclone `copy`, `sync`, `check`, and `bisync` commands and links to the separate workflow for configuring and recovering a bidirectional synchronization pair. The examples use PowerShell and assume that one side of the synchronization is a Google Drive remote. + +Rclone is a command-line file-management program that supports cloud storage providers, object-storage platforms, local filesystems, and standard transfer protocols. + +[Download rClone](https://rclone.org/downloads) + +[Official rClone Documentation](https://rclone.org/docs/) + +!!! info "Version Context" + This document was reviewed against rclone `v1.75.0`. Older releases may not support every bisync flag documented here, and future versions may change some recovery behavior. + +!!! danger "Rclone Can Delete or Overwrite Data" + The `sync` and `bisync` commands can delete, replace, rename, or propagate the deletion of files. Always confirm the source and destination paths, maintain a separate backup or snapshot, and preview unfamiliar operations with `--dry-run` before allowing them to modify data. + + A dry run is a preview and is not a replacement for a backup. + +## Prepare the Command +Set the executable path in the PowerShell session before using the command examples. Replace the example path with the location of `rclone.exe` on the target machine. + +```powershell +$Rclone = "C:\Path\To\rclone.exe" +``` + +## Command Behavior +The correct command depends on the intended relationship between the source and destination. + +| **Command** | **Primary Behavior** | **Deletes Destination-Only Files** | **Direction** | +| :--- | :--- | :--- | :--- | +| `copy` | Adds or updates source files at the destination while retaining unrelated destination files | No | One-way | +| `sync` | Makes the destination match the source | Yes | One-way | +| `check` | Compares files without modifying either side | No | Read-only | +| `bisync` | Detects and propagates changes made on either side by comparing the current state against prior listings | Yes | Two-way | + +### Use `copy` for Additive Transfers +Use `copy` when you need to add or update files without deleting files that already exist only at the destination. + +```powershell +& $Rclone copy "Source" "Destination" --update --dry-run --verbose +``` + +The `--update` flag skips a source file when the corresponding destination file has a newer modification time. It does not create version history, and it does not prevent an older destination file from being replaced by a newer source file. + +After reviewing the dry-run output, repeat the operation without `--dry-run`: + +```powershell +& $Rclone copy "Source" "Destination" --update --verbose +``` + +### Use `sync` Only for Intentional Mirroring +Use `sync` when the destination must become a one-way mirror of the source. + +!!! danger "`sync` Deletes Destination-Only Files" + The `sync` command removes files from the destination when they do not exist in the source. This is true even when `--update` is present. + + The `--update` flag only prevents a newer destination file from being replaced by an older source file. It does not convert `sync` into an additive operation and does not protect destination-only files from deletion. + +Preview the operation first: + +```powershell +& $Rclone sync "Source" "Destination" --dry-run --verbose +``` + +Only remove `--dry-run` after confirming that every proposed copy, replacement, and deletion is intentional: + +```powershell +& $Rclone sync "Source" "Destination" --verbose +``` + +### Use `check` for Read-Only Comparison +The `check` command compares files on both sides without copying, replacing, or deleting them. + +```powershell +& $Rclone check "Source" "Destination" --combined "rclone-comparison.txt" --log-level INFO --log-file "rclone-check.log" +``` + +The combined report uses the following symbols: + +| **Symbol** | **Meaning** | +| :--- | :--- | +| `=` | The file exists on both sides and matches | +| `+` | The file exists only in the source | +| `-` | The file exists only in the destination | +| `*` | The same path exists on both sides, but the files differ | +| `!` | The file could not be read or compared | + +The `check` command compares files but does not report missing empty directories. + +## Configure Two-Way Synchronization +Use [Configure and Recover Rclone Bisync](<../../../workflows/Applications/Files and Collaboration/Configure and Recover Rclone Bisync.md>) for initialization, normal runs, state handling, and recovery. + +## Related Documentation +- [Related Files and Collaboration Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/batch/robocopy.md b/reference/Applications/Files and Collaboration/Robocopy Command Reference.md similarity index 88% rename from scripts/batch/robocopy.md rename to reference/Applications/Files and Collaboration/Robocopy Command Reference.md index f6d75de..474fe78 100644 --- a/scripts/batch/robocopy.md +++ b/reference/Applications/Files and Collaboration/Robocopy Command Reference.md @@ -6,12 +6,14 @@ tags: - Windows --- -Robocopy is a useful tool that can be leveraged to copy files and folders from one location to another (e.g. Over the network to another server) without losing file and folder ACLs (permissions / ownership data). +## Purpose +Robocopy is a useful tool that can be leveraged to copy files and folders from one location to another (e.g. Over the network to another server) without losing file and folder ACLs (permissions / ownership data). !!! warning "Run as Domain Admin" When you run Robocopy, especially when transferring data across the network to another remote server, you need to be sure to run the command prompt under the session of a domain admin. Secondly, it needs to be ran as an administrator to ensure the command is successful. This can be done by going to the start menu and typing "**Command Prompt**" > **Right Clicking** > "**Run as Administrator**" while logged in as a domain administrator. An example of using Robocopy is below, with a full breakdown: + ```powershell robocopy "E:\Source" "Z:\Destination" /Z /B /R:5 /W:5 /MT:4 /COPYALL /E ``` @@ -26,11 +28,15 @@ robocopy "E:\Source" "Z:\Destination" /Z /B /R:5 /W:5 /MT:4 /COPYALL /E - `/E` : Directs Robocopy to include all subdirectories in the copy operation, ensuring even empty directories are replicated in the destination. !!! tip "Usage of Administrative Shares" - Whenever dealing with copying data from one server to another, try to leverage "Administrative Shares", also referred to as "Default Shares". These exist in such a way that, if the server exists in a Windows-based domain, you can type something like `\\SERVER\C$` or `\\SERVER\E$` to access files and bypass most file access restrictions (ACLs). This generally only applies to read-access, write-access may be denied in some circumstances. - - An adjusted example can be seen below to account for this usage. + Whenever dealing with copying data from one server to another, try to leverage "Administrative Shares", also referred to as "Default Shares". These exist in such a way that, if the server exists in a Windows-based domain, you can type something like `\\SERVER\C$` or `\\SERVER\E$` to access files and bypass most file access restrictions (ACLs). This generally only applies to read-access, write-access may be denied in some circumstances. + + An adjusted example can be seen below to account for this usage. **This example assumes you are running robocopy from the destination computer**. **Remember**: You are always **PULLING** data with administrative shares, not pushing it, the source should be the administrative share, and the destination should be local (in this example). There are scenarios where you can move data between two network shares, but its best (and cleaner) to always have a remote/local relationship in the transfer. + ```powershell robocopy "\\SERVER\E$\SOURCE" "E:\DESTINATION" /Z /B /R:5 /W:5 /MT:4 /COPYALL /E - ``` \ No newline at end of file + ``` + +## Related Documentation +- [Related Files and Collaboration Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/Applications/Files and Collaboration/index.md b/reference/Applications/Files and Collaboration/index.md new file mode 100644 index 0000000..5310755 --- /dev/null +++ b/reference/Applications/Files and Collaboration/index.md @@ -0,0 +1,35 @@ +--- +tags: + - File Services + - Nextcloud + - Synchronization +--- + +# Files and Collaboration +## Purpose +Find the service that owns the data, then choose its integration, permissions, or transfer procedure. Distinguish an additive copy, a one-way mirror, and a two-way synchronization before running a transfer tool. + +## Includes +- Nextcloud and office integration +- Windows DFS and permission reports +- File-copy and synchronization tools + +## Applications and Office Integration +- [Nextcloud AIO](<../../../deployments/Applications/Files and Collaboration/Nextcloud AIO.md>) — Follow the AIO deployment and its integration requirements. +- [Separate Nextcloud Deployment](<../../../deployments/Applications/Files and Collaboration/Nextcloud.md>) — Consult the alternative container example. +- [Collabora](<../../../deployments/Applications/Files and Collaboration/Collabora Code Server.md>) — Connect the office service to the matching Nextcloud installation. +- [OnlyOffice](<../../../deployments/Applications/Files and Collaboration/OnlyOffice EE.md>) — Review the separately documented office-service option. +- [Upload to a Nextcloud Share](<../../../scripts/Applications/Files and Collaboration/Upload Data to a Nextcloud Share.md>) — Use the PowerShell or Bash example for a shared upload destination. + +## Windows File Services +- [Deploy DFS](<../../../deployments/Applications/Files and Collaboration/Windows Server/DFS Namespaces with Replication.md>) — Build the namespaces and replication before using the reports. +- [Report DFS Configuration](<../../../scripts/Applications/Files and Collaboration/DFS/Report DFS Namespaces and Replication.md>) — Inspect namespace targets and replication configuration. +- [Report DFS Backlog](<../../../scripts/Applications/Files and Collaboration/DFS/Report DFS Replication Backlog.md>) — Check directional replication progress. +- [SMB Share Permissions](<../../../scripts/Applications/Files and Collaboration/SMB/Report SMB Share Permissions.md>) — Report the permissions assigned at the share layer. +- [NTFS Permissions](<../../../scripts/Applications/Files and Collaboration/SMB/Report NTFS Permissions Across Shares.md>) — Report filesystem ACLs beneath the shared paths. + +## Choose a Transfer Procedure +- [Robocopy]() — Review the Windows copy and mirror examples and their permission handling. +- [Rclone Commands]() — Compare copy, sync, check, and bisync behavior. +- [Rclone Bisync](<../../../workflows/Applications/Files and Collaboration/Configure and Recover Rclone Bisync.md>) — Configure a persistent two-way synchronization pair and its recovery state. +- [Netcat Transfer](<../../../workflows/Applications/Files and Collaboration/Transfer Files with Netcat.md>) — Review the dedicated transfer example and its transport limitations. diff --git a/reference/Applications/index.md b/reference/Applications/index.md new file mode 100644 index 0000000..5338dde --- /dev/null +++ b/reference/Applications/index.md @@ -0,0 +1,23 @@ +--- +tags: + - Applications + - Reference + - Documentation +--- + +# Applications +## Purpose +Find applications by the service they provide, then continue to their deployment, authentication, data, and maintenance documentation. + +## Includes +- Choose the Mailcow/PMG, iRedMail, or Exchange documentation that matches the environment. +- Connect Nextcloud, office integration, DFS, permissions, and file-transfer tools. +- Start with Home Assistant, then follow the camera and device integration links. + +## Find the Right Document +- [Email]() — Choose the Mailcow/PMG, iRedMail, or Exchange documentation that matches the environment. +- [Files and Collaboration]() — Connect Nextcloud, office integration, DFS, permissions, and file-transfer tools. +- [Home Automation](<../../deployments/Applications/Home Automation/HomeAssistant.md>) — Start with Home Assistant, then follow the camera and device integration links. +- [Monitoring and Notifications](<../../deployments/Applications/Monitoring/Gatus.md>) — Find endpoint monitoring and the separately documented ntfy notification service. +- [Game Hosting](<../../deployments/Applications/Gaming and Media/Pterodactyl.md>) — Find the control panel alongside individual server deployments. +- [Application Deployments](<../../deployments/Applications/index.md>) — Browse the remaining asset-management, communication, dashboard, and utility services. diff --git a/reference/Automation/AWX/Credential Configuration Examples.md b/reference/Automation/AWX/Credential Configuration Examples.md new file mode 100644 index 0000000..ab66189 --- /dev/null +++ b/reference/Automation/AWX/Credential Configuration Examples.md @@ -0,0 +1,48 @@ +--- +tags: + - Ansible + - Automation +--- + +## Purpose +Record AWX credential examples for Linux and Windows targets. The examples retain their original domain context and must be reconciled with the authentication method used by the target environment. + +!!! info "Recorded Credential Examples" + These examples use the `MOONGATE.LOCAL` domain and record Kerberos limitations from that setup. The separate [AWX Kerberos implementation](<../../../workflows/Automation/AWX/AWX Kerberos Implementation.md>) describes a `BUNNY-LAB.IO` configuration. Confirm which environment applies before using either example. + +## Windows-based Credentials +### NTLM +NTLM-based authentication is not exactly the most secure method of remotely running playbooks on Windows devices, but it is still encrypted using SSL certificates created by the device itself when provisioned correctly to enable WinRM functionality. + +```text title="(NTLM) nicole.rappe@MOONGATE.LOCAL" +Credential Type: Machine +Username: nicole.rappe@MOONGATE.LOCAL +Password: +Privilege Escalation Method: runas +Privilege Escalation Username: nicole.rappe@MOONGATE.LOCAL +``` + +### Kerberos +Kerberos-based authentication is generally considered the most secure method of authentication with Windows devices, but can be trickier to set up since it requires additional setup inside of AWX in the cluster for it to function properly. The separately documented AWX Kerberos implementation describes a different environment. + +```text title="(Kerberos WinRM) nicole.rappe" +Credential Type: Kerberos WinRM +Username: nicole.rappe +Password: +Kerberos Realm (Domain): MOONGATE.LOCAL +``` + +## Linux-based Credentials +```text title="(LINUX) nicole" +Credential Type: Machine +Username: nicole +Password: +Privilege Escalation Method: sudo +Privilege Escalation Username: root +``` + +!!! note "Note" + `WinRM / Kerberos` based credentials do not currently work as-expected. That limitation belongs to this recorded example; consult the linked Kerberos workflow for the separate implementation. + +## Related Documentation +- [Related AWX Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/workflows/operations/automation/ansible/credentials/Custom Credential Types/WinRM.md b/reference/Automation/AWX/Custom Kerberos WinRM Credential.md similarity index 59% rename from workflows/operations/automation/ansible/credentials/Custom Credential Types/WinRM.md rename to reference/Automation/AWX/Custom Kerberos WinRM Credential.md index 7f96c9d..4ae52c8 100644 --- a/workflows/operations/automation/ansible/credentials/Custom Credential Types/WinRM.md +++ b/reference/Automation/AWX/Custom Kerberos WinRM Credential.md @@ -5,10 +5,10 @@ tags: - Automation --- -# WinRM (Kerberos) -**Name**: "Kerberos WinRM" +## Purpose +Record the input and injector definitions for the custom AWX Kerberos WinRM credential type. -```jsx title="Input Configuration" +```yaml title="Input Configuration" fields: - id: username type: string @@ -26,10 +26,13 @@ required: - krb_realm ``` -```jsx title="Injector Configuration" +```yaml title="Injector Configuration" extra_vars: ansible_user: '{{ username }}' ansible_password: '{{ password }}' ansible_winrm_transport: kerberos ansible_winrm_kerberos_realm: '{{ krb_realm }}' -``` \ No newline at end of file +``` + +## Related Documentation +- [Related AWX Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/workflows/operations/automation/ansible/inventories/overview.md b/reference/Automation/AWX/Inventory Structure and Variables.md similarity index 74% rename from workflows/operations/automation/ansible/inventories/overview.md rename to reference/Automation/AWX/Inventory Structure and Variables.md index c4f93bb..80a7ecf 100644 --- a/workflows/operations/automation/ansible/inventories/overview.md +++ b/reference/Automation/AWX/Inventory Structure and Variables.md @@ -4,10 +4,10 @@ tags: - Automation --- -# Host Inventories -When you are deploying playbooks, you target hosts that exist in "Inventories". These inventories consist of a list of hosts and their corresponding IP addresses, as well as any host-specific variables that may be necessary to declare to run the playbook. You can see an example inventory file below. +## Purpose +Explain how AWX inventories describe hosts, groups, and connection variables. Use the lab inventory for environment-specific host records. -Keep in mind the "Group Variables" section varies based on your environment. NTLM is considered insecure, but may be necessary when you are interacting with Windows servers that are not domain-joined. Otherwise you want to use Kerberos authentication. This is outlined more in the [AWX Kerberos Implementation](../awx/awx-kerberos-implementation.md#job-template-inventory-examples) documentation. +Keep in mind the "Group Variables" section varies based on your environment. NTLM is considered insecure, but may be necessary when you are interacting with Windows servers that are not domain-joined. Otherwise you want to use Kerberos authentication. This is outlined more in the [AWX Kerberos Implementation](<../../../workflows/Automation/AWX/AWX Kerberos Implementation.md#job-template-and-inventory-examples>) documentation. !!! note "Inventory Data Relationships" An inventory file consists of hosts, groups, and variables. A host belongs to a group, and a group can have variables configured for it. If you run a playbook / job template against a host, it will assign the variables associated to the group that host belongs to (if any) during runtime. @@ -39,3 +39,5 @@ ansible_winrm_transport=ntlm ansible_winrm_server_cert_validation=ignore ``` +## Related Documentation +- [Related AWX Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/workflows/operations/automation/ansible/templates/overview.md b/reference/Automation/AWX/Job Template Configuration.md similarity index 71% rename from workflows/operations/automation/ansible/templates/overview.md rename to reference/Automation/AWX/Job Template Configuration.md index 4407db9..c92e0a7 100644 --- a/workflows/operations/automation/ansible/templates/overview.md +++ b/reference/Automation/AWX/Job Template Configuration.md @@ -4,10 +4,10 @@ tags: - Automation --- -# Templates -Templates are basically pre-constructed groups of devices, playbooks, and credentials that perform a specific kind of task against a predefined group of hosts or device inventory. +## Purpose +Record the fields and variables used by the example AWX job template that deploys a Hyper-V guest. -```jsx title="Deploy Hyper-V VM" +```text title="Deploy Hyper-V VM" Name: Deploy Hyper-V VM Inventory: (NTLM) MOON-HOST-01 Playbook: playbooks/Windows/Hyper-V/Deploy-VM.yml @@ -24,4 +24,7 @@ vm_memory: "8589934592" #Measured in Bytes (e.g. 8GB) vm_storage: "68719476736" #Measured in Bytes (e.g. 64GB) iso_path: "C:\\ubuntu-22.04-live-server-amd64.iso" vm_folder: "C:\\Virtual Machines\\{{ vm_name_fact }}" -``` \ No newline at end of file +``` + +## Related Documentation +- [Related AWX Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/workflows/operations/automation/ansible/playbooks/playbooks.md b/reference/Automation/AWX/Playbook Catalog.md similarity index 83% rename from workflows/operations/automation/ansible/playbooks/playbooks.md rename to reference/Automation/AWX/Playbook Catalog.md index c15c7f9..eb7fe56 100644 --- a/workflows/operations/automation/ansible/playbooks/playbooks.md +++ b/reference/Automation/AWX/Playbook Catalog.md @@ -4,15 +4,15 @@ tags: - Automation --- +## Purpose +This is an indexed list of Ansible Playbooks / Workflows that I have developed to deploy and manage various aspects of my lab environment. The list is not dynamically updated, so it may sometimes be out-of-date. + !!! warning "DOCUMENT UNDER CONSTRUCTION" This document is a "scaffold" document. It is missing significant portions of several sections and should not be read with any scrutiny until it is more feature-complete down-the-road. Come back later and I should have added more to this document hopefully by then. -**Purpose**: -This is an indexed list of Ansible Playbooks / Workflows that I have developed to deploy and manage various aspects of my lab environment. The list is not dynamically updated, so it may sometimes be out-of-date. - ## Linux Playbooks ### Deployments -Deployment playbooks are meant to be playbooks (or a series of playbooks forming a "Workflow Job Template") that deploy a server or piece of software. +Deployment playbooks are meant to be playbooks (or a series of playbooks forming a "Workflow Job Template") that deploy a server or piece of software. - Authentik - [1-Authentik-Bootstrapper.yml](https://git.bunny-lab.io/GitOps/awx.bunny-lab.io/src/branch/main/playbooks/Linux/Deployments/Authentik/1-Authentik-Bootstrapper.yml) @@ -29,13 +29,14 @@ Deployment playbooks are meant to be playbooks (or a series of playbooks forming - PrivacyIDEA - [privacyIDEA.yml](https://git.bunny-lab.io/GitOps/awx.bunny-lab.io/src/branch/main/playbooks/Linux/Deployments/privacyIDEA.yml) - Rancher RKE2 Kubernetes Cluster - - [PLACEHOLDER]() - - [PLACEHOLDER]() - - [PLACEHOLDER]() - - [PLACEHOLDER]() - - [PLACEHOLDER]() + - PLACEHOLDER (not documented) + - PLACEHOLDER (not documented) + - PLACEHOLDER (not documented) + - PLACEHOLDER (not documented) + - PLACEHOLDER (not documented) + ### Kerberos -This playbook is designed to be chain-loaded before any playbooks that involve interacting with Active Directory Domain-Joined Windows Devices. It establishes a connection with Active Directory using domain credentials, sets up a keytab file (among other things), and makes it so the execution environment that the subsequent jobs are running in are able to run against windows devices. This ensures the connection is encrypted the entire time the playbooks are running instead of using lower-security authentication methods like NTLM, which don't even always work in most circumstances. You can find more information in the [Kerberos Authentication](../awx/awx-kerberos-implementation.md#kerberos-implementation) section of the AWX documentation. `It does require additional setup prior to running the playbook.` +This playbook is designed to be chain-loaded before any playbooks that involve interacting with Active Directory Domain-Joined Windows Devices. It establishes a connection with Active Directory using domain credentials, sets up a keytab file (among other things), and makes it so the execution environment that the subsequent jobs are running in are able to run against windows devices. This ensures the connection is encrypted the entire time the playbooks are running instead of using lower-security authentication methods like NTLM, which don't even always work in most circumstances. You can find more information in the [Kerberos Authentication](<../../../workflows/Automation/AWX/AWX Kerberos Implementation.md#kerberos-implementation>) section of the AWX documentation. `It does require additional setup prior to running the playbook.` - [Establish_Kerberos_Connection.yml](https://git.bunny-lab.io/GitOps/awx.bunny-lab.io/src/branch/main/playbooks/Linux/Establish_Kerberos_Connection.yml) @@ -46,17 +47,19 @@ This playbook is designed to be chain-loaded before any playbooks that involve i Security playbooks do things like secure devices with additional auditing functionality, login notifications, enforcing SSH certificate-based authentication, things of that sort. - Install SSH Public Key Authentication - - [PLACEHOLDER]() + - PLACEHOLDER (not documented) - SSH Login Notifications - - [PLACEHOLDER]() + - PLACEHOLDER (not documented) ## Windows Playbooks ### Deployments -Deployment playbooks are meant to be playbooks (or a series of playbooks forming a "Workflow Job Template") that deploy a server or piece of software. +Deployment playbooks are meant to be playbooks (or a series of playbooks forming a "Workflow Job Template") that deploy a server or piece of software. - Hyper-V - Deploy GuestVM - - [PLACEHOLDER]() + - PLACEHOLDER (not documented) - Query Active Directory Domain Computers - - [PLACEHOLDER]() + - PLACEHOLDER (not documented) - Install BGInfo - - [PLACEHOLDER]() + - PLACEHOLDER (not documented) +## Related Documentation +- [Related AWX Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/Automation/AWX/Projects and Source Control.md b/reference/Automation/AWX/Projects and Source Control.md new file mode 100644 index 0000000..d4a78d1 --- /dev/null +++ b/reference/Automation/AWX/Projects and Source Control.md @@ -0,0 +1,21 @@ +--- +tags: + - AWX + - Gitea + - Automation +--- + +## Purpose +Understand how an AWX project supplies playbooks and inventory files from source control. Maintain the Gitea connection settings in the connection workflow so the project reference does not become a second configuration source. + +## Project Relationships +A project identifies the repository and source-control credential. An inventory source can consume an inventory file from that project, and a job template selects a playbook from the project. + +## Configure the Connection +[Connect AWX to Gitea](<../../../workflows/Automation/AWX/Connect AWX to Gitea.md>) contains the source URL, credential fields, inventory source, and overwrite behavior. + +## Continue to Job Execution +[The AWX guide]() connects inventory structure, credential examples, and job-template configuration. + +## Related Documentation +- [Related AWX Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/Automation/AWX/index.md b/reference/Automation/AWX/index.md new file mode 100644 index 0000000..4bafe78 --- /dev/null +++ b/reference/Automation/AWX/index.md @@ -0,0 +1,36 @@ +--- +tags: + - AWX + - Ansible + - Automation +--- + +# AWX +## Purpose +Follow AWX from its Kubernetes deployment through source control, inventory, credentials, and job execution. Check the environment context on older Minikube and credential examples before combining them with the operator deployment. + +## Includes +- Controller deployment and upgrades +- Projects, inventories, credentials, and templates +- Gitea and Windows authentication integration + +## Build the Controller +- [Rancher RKE2](<../../../deployments/Containers/Kubernetes/Rancher RKE2.md>) — Prepare the cluster required by the AWX Operator procedure. +- [AWX Operator](<../../../deployments/automation/AWX/AWX Operator.md>) — Deploy the controller into the documented cluster. +- [Minikube Example](<../../../deployments/automation/AWX/AWX in Minikube.md>) — Consult the separate deployment approach and its recorded assumptions. + +## Connect the Automation Objects +A project supplies repository content. An inventory identifies targets and variables. Credentials provide authentication, and a job template combines these objects with a playbook. + +- [Connect AWX to Gitea](<../../../workflows/Automation/AWX/Connect AWX to Gitea.md>) — Create the source credential, project, and inventory source together. +- [Projects and Source Control]() — Understand the project role without maintaining a second copy of its connection settings. +- [Inventory Structure]() — Understand host groups and variables before changing the lab inventory. +- [Credential Examples]() — Review the recorded environment and authentication limitations. +- [Job Templates]() — Connect the project, inventory, playbook, and credentials. +- [Lab Inventory](<../../Lab Map/Homelab Server Inventory.md>) — Locate the recorded hosts and inventory groups. + +## Run Against Windows Targets +- [Prepare Windows Targets](<../../../workflows/Identity and Certificates/Windows/Enable WinRM over HTTPS.md>) — Configure the remote-management endpoint used by the automation examples. +- [Configure AWX Kerberos](<../../../workflows/Automation/AWX/AWX Kerberos Implementation.md>) — Follow the execution-environment and FQDN requirements recorded for this implementation. +- [Custom WinRM Credential]() — Find the credential input and injector definitions. +- [Playbook Catalog]() — Find existing repository playbooks and the catalog completeness notes. diff --git a/reference/Automation/Gitea Configuration Delivery.md b/reference/Automation/Gitea Configuration Delivery.md new file mode 100644 index 0000000..09f7b84 --- /dev/null +++ b/reference/Automation/Gitea Configuration Delivery.md @@ -0,0 +1,23 @@ +--- +tags: + - Gitea + - GitOps + - Automation +--- + +## Purpose +Choose the documented Gitea configuration-delivery approach that matches the target host and execution environment. The examples preserve distinct implementations rather than a single interchangeable runner configuration. + +## Documented Approaches +- [Docker Runner](<../../workflows/Automation/Gitea/Deliver Configuration with a Docker Runner.md>) — The May 2025 example runs jobs inside the runner container and writes to a bind-mounted destination. +- [Zensical Host Runner](<../../workflows/Automation/Gitea/Publish Zensical Documentation with a Host Runner.md>) — The Zensical-specific example uses a host service account, watchdog permissions, and `/srv/zensical/docs`. +- [Git Repo Updater](<../../deployments/Containers/Docker/Git Repo Updater.md>) — The earlier polling-container approach retains its deployment instructions and canonical watcher script. +- [Why I Adopted Runners](<../../blog/posts/05-16-2025 Learning to Leverage Gitea Runners.md>) — Read the dated account of the watcher limitations and runner experiment. + +## Follow the Destination Service +- [Zensical Deployment](<../../deployments/automation/Documentation/Zensical.md>) — Prepare the service and destination before applying the host-runner workflow. +- [Traefik Configuration](<../../deployments/Networking and Access/Reverse Proxies/Traefik.md>) — Identify the dynamic configuration destination before selecting a delivery method. +- [ntfy Notifications](<../../deployments/Applications/Monitoring/Ntfy.md>) — Find the notification service used by the runner examples. + +## Related Documentation +- [Related Automation Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/Automation/index.md b/reference/Automation/index.md new file mode 100644 index 0000000..c448aa8 --- /dev/null +++ b/reference/Automation/index.md @@ -0,0 +1,23 @@ +--- +tags: + - Automation + - Reference + - Documentation +--- + +# Automation +## Purpose +Connect source control, automation controllers, managed hosts, and configuration delivery. Use the documented execution environment and authentication method for each workflow. + +## Includes +- Follow deployment, source control, inventory, credentials, and job execution. +- Build the Puppet environment and its Gitea configuration integration. +- Use the related remote-execution tooling and target prerequisites. + +## Find the Right Document +- [AWX]() — Follow deployment, source control, inventory, credentials, and job execution. +- [Puppet](<../../deployments/automation/Puppet/Puppet.md>) — Build the Puppet environment and its Gitea configuration integration. +- [Puppet Bolt](<../../deployments/automation/Puppet/Puppet Bolt.md>) — Use the related remote-execution tooling and target prerequisites. +- [Gitea](<../../deployments/automation/Gitea/Gitea.md>) — Deploy the source-control service used by controllers and runners. +- [Choose a Configuration Delivery Method]() — Compare the documented watcher, Docker runner, and Zensical host runner. +- [FOG Imaging](<../../deployments/automation/FOG Project/Deploy FOG Project.md>) — Connect the imaging deployment to the matching DHCP/PXE configuration. diff --git a/workflows/operations/Backups and DR/Veeam Backup Replication/Core Veeam Concepts.md b/reference/Backup and Recovery/Veeam Concepts.md similarity index 95% rename from workflows/operations/Backups and DR/Veeam Backup Replication/Core Veeam Concepts.md rename to reference/Backup and Recovery/Veeam Concepts.md index ad46d92..e6c1b6e 100644 --- a/workflows/operations/Backups and DR/Veeam Backup Replication/Core Veeam Concepts.md +++ b/reference/Backup and Recovery/Veeam Concepts.md @@ -5,41 +5,48 @@ tags: - Disaster Recovery --- -**Purpose**: +## Purpose The purpose of this document is to explain the core concepts / terminology of things seen in Veeam Backup & Replication from a relatively high-level. It's more of a quick-reference guide than a formal education. ## Backup Jobs Backup jobs take many forms, but the most common are explained in more detail below. Note that this is not an exhaustive list of the different kinds of backup jobs, just the ones I am currently most familiar with. -- **Backup**: This is the simplest of the backup job options. A "Backup" backup job will take a backup of a workstation, server, File Server, specific local files and folders on a device, or a GuestVM running in a hypervisor such as Hyper-V, VMWare ESXi, or ProxmoxVE. -- **Backup Copy**: - - This is when you make a copy of backup data stored on the Veeam server, and send it somewhere else, such as an off-site "Service Provider" such as Veeam partners. +- **Backup**: This is the simplest of the backup job options. A "Backup" backup job will take a backup of a workstation, server, File Server, specific local files and folders on a device, or a GuestVM running in a hypervisor such as Hyper-V, VMWare ESXi, or ProxmoxVE. +- **Backup Copy**: + - This is when you make a copy of backup data stored on the Veeam server, and send it somewhere else, such as an off-site "Service Provider" such as Veeam partners. - You can also send backup copies to local drives, SMB network shares, NFS shares, File Servers, pretty much anywhere you can send normal backups, but with the key difference being the data is originating from the Veeam backup server itself instead of the original server/VM. -- **SureBackup**: This is where things get a little more complex. SureBackup is where you effectively "Verify" your backups by spinning them up inside of a lab environment. While they are spun up, they are checked to see if they fully boot, they can have antivirus scans, ransomware scans, custom scripts executed, and validate the integrity of the backups. The general core components are listed below: +- **SureBackup**: This is where things get a little more complex. SureBackup is where you effectively "Verify" your backups by spinning them up inside of a lab environment. While they are spun up, they are checked to see if they fully boot, they can have antivirus scans, ransomware scans, custom scripts executed, and validate the integrity of the backups. The general core components are listed below: - **Virtual Lab**: The virtual lab is a virtual machine environment that you set up for Veeam to leverage to spin up backups on a hypervisor that you configure, such as a remote Hyper-V server in the same building, or perhaps if you have Hyper-V locally installed on the same server as Veeam itself, you would configure the virtual lab's hypervisor to point to `127.0.0.1` or `localhost`. - The virtual lab will have its own unique virtual networking for the VMs to communicate on, so they don't conflict with the production servers/VMs. - **Application Groups**: Application groups are defined groups of devices that need to be running when the backups are being validated. For example, in my homelab, I have an application group named `Domain Controllers`, and I put `LAB-DC-01` and `LAB-DC-02` into that application group. I use this as the application group associated with the Virtual Lab because most of my services are authenticated with Active Directory, and if the DCs were missing during backup verification, a variety of issues would ensue. When the Backup Verification Lab (Virtual Lab) is launched on the targeted hypervisor, it spins up the application group devices from backups first, ensuring they are running and functional, before the virtual lab starts verifying backup objects designated in the "Linked Jobs", seen in the next section. - **Linked Jobs**: These are the "Backup Jobs" you want to verify in in the virtual lab mentioned above. If you have a large backup job with a bunch of machines you don't want verified, you can configure "Exclusions" in the SureBackup job settings to exclude those objects/devices from verification. + ## Replication Jobs As the name states, Veeam Backup & Replication can also handle replicating Servers/VMs from either their original locations or from a recent backup and push them into a hypervisor for rapid failover/failback functionality. Very useful for workloads that need to be spun up nearly immediately due to strict RTO requirements. There are some additional notes regarding replication seen below. !!! warning "Orchestrate Replication & Failover via Veeam, not the Hypervisor" You want to coordinate anything replication-wise directly in Veeam Backup & Replication, not directly on the hypervisor itself. While you can do this, it is not only slower, but does not give you the option to failback replicas back into production if you spin up a replica directly on its hypervisor. - -- **Replication Restore Points**: Similar to backups, replicas can have multiple restore points associated with them, so you have more than one option when spinning up a replica in a hypervisor. + +- **Replication Restore Points**: Similar to backups, replicas can have multiple restore points associated with them, so you have more than one option when spinning up a replica in a hypervisor. - **Planned Failover**: A planned failover is when you are scheduling the hypervisor to be offline and simply don't have enough resources to live-migrate it to another cluster host, or you might not even have a virtualization cluster to work with in the first place. In cases like this, a "Planned Failover" tells Veeam to make a fresh replica right now, then shuts down the production VM on its hypervisor, and spins up the replica on the replica server. (If you installed Hyper-V on the Veeam server, it would spin up the replica on the backup server itself). - A "Planned Failover" allows you to perform a "**Failback to Production**" when the failover event has concluded. This means that while the production VM was offline and the replica took over the production load, any changes made such as new files added, applications installed, etc will be replicated back to the production VM when the replica is "Failed back to Production". **This is the ideal choice in most circumstances**. - **Failover Now**: Failover now means that the production hypervisor is likely completely dead, and may need to be re-built, or you simply dont need to replicate changes back to production hypervisor after the failover event has concluded, such as on a low-priority print server. Any changes made while the replica is operational will be completely lost when the production VM is turned back on again or a restore is pushed back onto a new hypervisor. + ## Backup Infrastructure ### Backup Repository A backup repository is simply a destination to send the backups or backup copies. It can be anything from direct attached storage to a SMB file share on a NAS, or even off-site storage like Backblaze B2 or Amazon S3. - If you use object storage like Backblaze B2 or Amazon S3, you can configure an "Immutability Period" for backups that are sent to these destinations, meaning if your backup server was hit by ransomware or a malicious actor, neither they nor you could delete the backups in the off-site storage such as Backblaze B2 until the immutability period had passed, such as 7 days, 30 days, or however long you configured. - You can adjust the immutability period after-the-fact, but backups that have already been pushed to a backup repository will be immutable for the time period configured when they were originally uploaded, and attempts to delete them will tell you when you are allowed to delete them. You won't be able to delete them even from Amazon or Backblaze's own internal tools / websites during this immutability period. + ### Backup Proxy A backup "proxy" simply refers to a machine that is running the "**Veeam Backup Transport**" agent on it. The Veeam Backup & Replication server installs a proxy onto itself, but it also deploys proxies onto workstations, servers, and hypervisors. These proxies are how the "Veeam Backup & Replication Console" interacts with the devices and performs backups and restores. + ### Service Provider Service Providers are not the same as cloud storage providers such as Backblaze B2, Amazon S3, etc. Service Providers are Veeam "partners" who manage, maintain, and deploy Veeam backup appliances at client environments, as well as providing support to clients within the Veeam ecosystem. You can also use Service Providers as a cloud backup destination in Veeam Backup & Replication for off-site backups. ## Misc Terminology - **Unstructured Data**: This refers to a device such as a windows or linux server that you can use WinRM or SSH to access, and want to backup specific files and folders without backing up the entire device / VM. This is useful in cases where you cannot install a Veeam Agent or the operating system is unsupported by Veeam, or if the device is not operating under a hypervisor, such as a bare-metal server. - - When you add a device to Veeam's "Inventory" via the "Unstructured Data" section, if you want to perform backups on the device, you will have to make a special backup job under "**Backups > File Server**", because Veeam will treat the unstructured data as a file server. \ No newline at end of file + - When you add a device to Veeam's "Inventory" via the "Unstructured Data" section, if you want to perform backups on the device, you will have to make a special backup job under "**Backups > File Server**", because Veeam will treat the unstructured data as a file server. + +## Related Documentation +- [Related Backup and Recovery Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/workflows/operations/Backups and DR/Veeam Backup Replication/Retention Best Practices.md b/reference/Backup and Recovery/Veeam Retention Policy Example.md similarity index 74% rename from workflows/operations/Backups and DR/Veeam Backup Replication/Retention Best Practices.md rename to reference/Backup and Recovery/Veeam Retention Policy Example.md index 8d285f8..dac52aa 100644 --- a/workflows/operations/Backups and DR/Veeam Backup Replication/Retention Best Practices.md +++ b/reference/Backup and Recovery/Veeam Retention Policy Example.md @@ -5,9 +5,12 @@ tags: - Disaster Recovery --- -**Purpose**: +## Purpose This is meant as a high-level generally-speaking best practice retention policy in most use-cases. This document will generally be pretty bare-bones, but the general idea is the following advanced GFS retention period is generally configured on backup copy jobs, specifically ones that have off-site backups, but can also be used for local backup repositories. +!!! info "Example Retention Policy" + This is the recorded policy example and its assumptions. Select retention for the actual workload and recovery requirements before applying these values. + Navigate to Jobs > Backup (or Backup Copy) > (Find a Backup Job) > Right-Click > Edit > Storage (or Target) > "**Keep Certain Full Backups for Archival Purposes**: Checked" > Click on the "**Configure**" button. Optional: Click the "**Save as Default**" button before clicking the "**OK**" button to make this default behavior for new backup jobs. @@ -21,4 +24,7 @@ Optional: Click the "**Save as Default**" button before clicking the "**OK**" bu !!! note "7 Daily Backups Assumption" This document assumes that you at (least) keep 7 daily backups in the normal backup schedule. Meaning **7 daily, 4 weekly, 3 monthly, and 1 yearly** backup is maintained at all times. - **7 daily, 4 weekly, 3 monthly, and 1 yearly** \ No newline at end of file + **7 daily, 4 weekly, 3 monthly, and 1 yearly** + +## Related Documentation +- [Related Backup and Recovery Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/Backup and Recovery/index.md b/reference/Backup and Recovery/index.md new file mode 100644 index 0000000..1702918 --- /dev/null +++ b/reference/Backup and Recovery/index.md @@ -0,0 +1,23 @@ +--- +tags: + - Backup and Recovery + - Reference + - Documentation +--- + +# Backup and Recovery +## Purpose +Find backup concepts, repository maintenance, and recovery dependencies. Select the procedure for the affected backup system and distinguish a backup restore from a replica or snapshot operation. + +## Includes +- Understand jobs, repositories, application groups, and replica terminology. +- Find the recorded GFS policy and its assumptions. +- Move agent ownership to a replacement backup server. + +## Find the Right Document +- [Veeam Concepts]() — Understand jobs, repositories, application groups, and replica terminology. +- [Retention Policy Example]() — Find the recorded GFS policy and its assumptions. +- [Adopt an Existing Backup Agent](<../../workflows/Backup and Recovery/Veeam/Backup Agent Takeover.md>) — Move agent ownership to a replacement backup server. +- [Manage Repository Capacity](<../../workflows/Backup and Recovery/Veeam/Manually Pruning Backups.md>) — Review the existing backup-chain removal procedure before deleting data. +- [Repair Gateway Certificate Trust](<../../workflows/Backup and Recovery/Veeam/Failed to Validate Certificates of Some Gateways.md>) — Follow the specific Cloud Connect certificate failure. +- [Hypervisor Recovery](<../Virtualization and Storage/index.md>) — Find the applicable Hyper-V replica or Proxmox migration procedure. diff --git a/reference/Containers/index.md b/reference/Containers/index.md new file mode 100644 index 0000000..e1015a8 --- /dev/null +++ b/reference/Containers/index.md @@ -0,0 +1,23 @@ +--- +tags: + - Containers + - Reference + - Documentation +--- + +# Containers +## Purpose +Prepare the Docker or Kubernetes environment used by application deployments, then follow the operating procedures for building, moving, and exposing workloads. + +## Includes +- Prepare the external network referenced by Docker deployment examples. +- Manage the Docker workloads described in the service pages. +- Prepare the Kubernetes cluster used by the AWX Operator guide. + +## Find the Right Document +- [Create the Docker Network](<../../deployments/Containers/Docker/Create the Docker Network.md>) — Prepare the external network referenced by Docker deployment examples. +- [Deploy Portainer](<../../deployments/Containers/Docker/Deploy Portainer.md>) — Manage the Docker workloads described in the service pages. +- [Deploy Rancher RKE2](<../../deployments/Containers/Kubernetes/Rancher RKE2.md>) — Prepare the Kubernetes cluster used by the AWX Operator guide. +- [Move a Compose Workload to Kubernetes](<../../workflows/Containers/Kubernetes/Migrating Docker Compose YML to K8s.md>) — Follow the ntfy example through conversion, service exposure, and Traefik integration. +- [Move Docker Containers](<../../workflows/Containers/Docker/Transfer Docker Containers Between Hosts.md>) — Transfer the existing container data and configuration between hosts. +- [Build and Publish an Image](<../../workflows/Containers/Docker/Build and Publish a Container Image.md>) — Use the documented container development workflow. diff --git a/deployments/services/authentication/Active Directory/LDAP Settings.md b/reference/Identity and Certificates/Active Directory/LDAP Connection Settings.md similarity index 70% rename from deployments/services/authentication/Active Directory/LDAP Settings.md rename to reference/Identity and Certificates/Active Directory/LDAP Connection Settings.md index 2e23139..aabc21c 100644 --- a/deployments/services/authentication/Active Directory/LDAP Settings.md +++ b/reference/Identity and Certificates/Active Directory/LDAP Connection Settings.md @@ -5,7 +5,8 @@ tags: - Authentication --- -**Purpose**: LDAP settings are used in various services from privacyIDEA to Nextcloud. This will outline the basic parameters in my homelab that are necessary to make it function. +## Purpose +LDAP settings are used in various services from privacyIDEA to Nextcloud. This will outline the basic parameters in my homelab that are necessary to make it function. | **Field** | **Value** | **Description** | | :--- | :--- | :--- | @@ -16,4 +17,7 @@ tags: | User / Bind DN | `CN=Nicole Rappe,CN=Users,DC=bunny-lab,DC=io` | This is the domain admin used to connect to LDAP | | User / Bind Password | `` | Domain Credentials for Domain Admin account | | Login Attribute | ` LDAP Filter: (&(&(|(objectclass=person))(|(|(memberof=CN=Domain Users,CN=Users,DC=bunny-lab,DC=io)(primaryGroupID=513))))(samaccountname=%uid)) ` | Used by Nextcloud | -| Login Attribute | `(sAMAccountName=*)(objectCategory=person)` | Used by PrivacyIDEA | \ No newline at end of file +| Login Attribute | `(sAMAccountName=*)(objectCategory=person)` | Used by PrivacyIDEA | + +## Related Documentation +- [Related Identity and Certificates Documentation](<../index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/Identity and Certificates/Keycloak Integrations.md b/reference/Identity and Certificates/Keycloak Integrations.md new file mode 100644 index 0000000..b677ff7 --- /dev/null +++ b/reference/Identity and Certificates/Keycloak Integrations.md @@ -0,0 +1,20 @@ +--- +tags: + - Keycloak + - OAuth2 + - Authentication +--- + +## Purpose +Choose the documented Keycloak integration for an application or reverse proxy after Keycloak is deployed. Application OAuth settings and reverse-proxy authentication serve different integration points. + +## Prepare Keycloak +[Deploy Keycloak](<../../deployments/Identity and Certificates/Keycloak/Deploy Keycloak.md>) includes the service and proxy-middleware configuration. + +## Configure an Application +- [Gitea OAuth2](<../../workflows/Identity and Certificates/Keycloak/Connect Gitea to Keycloak.md>) — Configure the documented application client. +- [Portainer OAuth2](<../../workflows/Identity and Certificates/Keycloak/Connect Portainer to Keycloak.md>) — Configure the documented Portainer integration. +- [Firefox Proxy Authentication](<../../deployments/Networking and Access/Remote Access/Firefox.md>) — Review the deployment that explains the proxy authentication flow. + +## Related Documentation +- [Related Identity and Certificates Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/Identity and Certificates/index.md b/reference/Identity and Certificates/index.md new file mode 100644 index 0000000..0dc232c --- /dev/null +++ b/reference/Identity and Certificates/index.md @@ -0,0 +1,23 @@ +--- +tags: + - Identity and Certificates + - Reference + - Documentation +--- + +# Identity and Certificates +## Purpose +Connect directory services, certificate trust, single sign-on, and application authentication. Start with the identity system involved, then follow the integration or maintenance procedure. + +## Includes +- Build the documented offline root, online subordinate CA, and publication point. +- Publish and monitor CRLs after the PKI exists. +- Provide the trust material needed by directory clients. + +## Find the Right Document +- [Deploy Certificate Services](<../../deployments/Identity and Certificates/Active Directory/Certificate Services.md>) — Build the documented offline root, online subordinate CA, and publication point. +- [Maintain Revocation Lists](<../../workflows/Identity and Certificates/Certificates/Publish and Maintain Certificate Revocation Lists.md>) — Publish and monitor CRLs after the PKI exists. +- [Export LDAPS Certificates](<../../workflows/Identity and Certificates/Certificates/Export Certificates for LDAPS Clients.md>) — Provide the trust material needed by directory clients. +- [LDAP Connection Settings]() — Locate the recorded directory connection parameters. +- [Keycloak Application Integrations]() — Choose reverse-proxy authentication or an application-specific OAuth integration. +- [Windows Remote Management](<../../workflows/Identity and Certificates/Windows/Enable WinRM over HTTPS.md>) — Prepare Windows targets for the AWX or Puppet Bolt workflows that reference WinRM. diff --git a/reference/infrastructure/hardware/Cluster Node 01/Disk Arrays.md b/reference/Lab Map/Hardware/Cluster Node 01 Disk Layout.md similarity index 80% rename from reference/infrastructure/hardware/Cluster Node 01/Disk Arrays.md rename to reference/Lab Map/Hardware/Cluster Node 01 Disk Layout.md index 882562e..d6116f7 100644 --- a/reference/infrastructure/hardware/Cluster Node 01/Disk Arrays.md +++ b/reference/Lab Map/Hardware/Cluster Node 01 Disk Layout.md @@ -14,4 +14,7 @@ This document is meant to help keep track disks and their associated serial numb | **Column 01** | **Column 02** | **Column 03** | **Column 04** | | :--- | :--- | :--- | :--- | | 240GB
`SN: 50026B77850B2DA9` | 240GB
`SN: 50026B7784D34038` | 240GB
`SN: 50026B7784E8A771` | 240GB
`SN: 50026B7784E8CB49` | -| 240GB
`SN: 50026B7784D3620D` | 240GB
`SN: 50026B7784D45C34` | 240GB
`SN: 50026B7784E8AC95` | 240GB
`SN: 50026B7784E8A983` | \ No newline at end of file +| 240GB
`SN: 50026B7784D3620D` | 240GB
`SN: 50026B7784D45C34` | 240GB
`SN: 50026B7784E8AC95` | 240GB
`SN: 50026B7784E8A983` | + +## Related Documentation +- [Related Lab Map Documentation](<../index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/infrastructure/hardware/Cluster Node 02/Disk Arrays.md b/reference/Lab Map/Hardware/Cluster Node 02 Disk Layout.md similarity index 85% rename from reference/infrastructure/hardware/Cluster Node 02/Disk Arrays.md rename to reference/Lab Map/Hardware/Cluster Node 02 Disk Layout.md index 1228964..df4daf8 100644 --- a/reference/infrastructure/hardware/Cluster Node 02/Disk Arrays.md +++ b/reference/Lab Map/Hardware/Cluster Node 02 Disk Layout.md @@ -20,4 +20,7 @@ This document is meant to help keep track disks and their associated serial numb | EMPTY
`SN: N/A` | EMPTY
`SN: N/A` | 600GB
`SN: 6XR31D8L` | | EMPTY
`SN: N/A` | EMPTY
`SN: N/A` | 600GB
`SN: 6XR33F2W` | | EMPTY
`SN: N/A` | EMPTY
`SN: N/A` | 600GB
`SN: 6XR32TFE` | -| EMPTY
`SN: N/A` | EMPTY
`SN: N/A` | EMPTY
`SN: N/A` | \ No newline at end of file +| EMPTY
`SN: N/A` | EMPTY
`SN: N/A` | EMPTY
`SN: N/A` | + +## Related Documentation +- [Related Lab Map Documentation](<../index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/infrastructure/hardware/Cluster Node 03/Disk Arrays.md b/reference/Lab Map/Hardware/Cluster Node 03 Disk Layout.md similarity index 78% rename from reference/infrastructure/hardware/Cluster Node 03/Disk Arrays.md rename to reference/Lab Map/Hardware/Cluster Node 03 Disk Layout.md index 4d18b8f..e223ea3 100644 --- a/reference/infrastructure/hardware/Cluster Node 03/Disk Arrays.md +++ b/reference/Lab Map/Hardware/Cluster Node 03 Disk Layout.md @@ -15,4 +15,7 @@ This document is meant to help keep track disks and their associated serial numb | :--- | :--- | :--- | :--- | | 8TB
`SN: X1P0A01NFDWF` | 8TB
`SN: WWZ1TJT2` | EMPTY
`SN: N/A` | EMPTY
`SN: N/A` | | 8TB
`SN: VRK6M6MK` | 8TB
`SN: Y1L0A0QQFDWF` | EMPTY
`SN: N/A` | EMPTY
`SN: N/A` | -| 8TB
`SN: VRK6XEMK` | EMPTY
`SN: N/A` | EMPTY
`SN: N/A` | 1TB
`SN: 2417E8A9A7C1` | \ No newline at end of file +| 8TB
`SN: VRK6XEMK` | EMPTY
`SN: N/A` | EMPTY
`SN: N/A` | 1TB
`SN: 2417E8A9A7C1` | + +## Related Documentation +- [Related Lab Map Documentation](<../index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/infrastructure/hardware/Storage Node 01 Truenas Core/Disk Arrays.md b/reference/Lab Map/Hardware/Storage Node 01 TrueNAS Core Disk Layout.md similarity index 88% rename from reference/infrastructure/hardware/Storage Node 01 Truenas Core/Disk Arrays.md rename to reference/Lab Map/Hardware/Storage Node 01 TrueNAS Core Disk Layout.md index eef1296..be9f394 100644 --- a/reference/infrastructure/hardware/Storage Node 01 Truenas Core/Disk Arrays.md +++ b/reference/Lab Map/Hardware/Storage Node 01 TrueNAS Core Disk Layout.md @@ -21,4 +21,7 @@ This document is meant to help keep track disks and their associated serial numb | 960GB
`SN: 50026B7785270194` | 960GB
`SN: 50026B77853B0F4C` | 960GB
`SN: 50026B7785270EB4` | 960G
`SN: 50026B76870D497D` | 960GB
`SN: 50026B778526FCC8` | | 960GB
`SN: 50026B778526F8F2` | 960GB
`SN: 50026B778526F8EB` | 960GB
`SN: 50026B778526FCFC` | 960GB
`SN: 50026B76870D47DF` | 960GB
`SN: 50026B778526F8EC` | | 960GB
`SN: 50026B7785270E2A` | 960GB
`SN: 50026B7785270653` | 960GB
`SN: 50026B778526FF67` | 960GB
`SN: 50026B7384228B63` | 960GB
`SN: 50026B778526FF4A` | -| 960GB
`SN: 50026B778526FFAA` | 960GB
`SN: 50026B778526FFC7` | 960GB
`SN: 50026B778526FF49` | 1TB
`SN: SI04T000311404D40` | EMPTY
`SN: N/A` | \ No newline at end of file +| 960GB
`SN: 50026B778526FFAA` | 960GB
`SN: 50026B778526FFC7` | 960GB
`SN: 50026B778526FF49` | 1TB
`SN: SI04T000311404D40` | EMPTY
`SN: N/A` | + +## Related Documentation +- [Related Lab Map Documentation](<../index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/infrastructure/hardware/Storage Node 02 Truenas Scale/Disk Arrays.md b/reference/Lab Map/Hardware/Storage Node 02 TrueNAS Scale Disk Layout.md similarity index 70% rename from reference/infrastructure/hardware/Storage Node 02 Truenas Scale/Disk Arrays.md rename to reference/Lab Map/Hardware/Storage Node 02 TrueNAS Scale Disk Layout.md index 6571699..f97284a 100644 --- a/reference/infrastructure/hardware/Storage Node 02 Truenas Scale/Disk Arrays.md +++ b/reference/Lab Map/Hardware/Storage Node 02 TrueNAS Scale Disk Layout.md @@ -14,4 +14,7 @@ This document is meant to help keep track disks and their associated serial numb | **Slot 01** | **Slot 02** | **Slot 03** | **Slot 04** | **Slot 05** | | :--- | :--- | :--- | :--- | :--- | -| 4TB
`SN: Z305ZNZM` | 4TB
`SN: Z305SPL8` | 4TB
`SN: Z3051AF8` | 4TB
`SN: Z305ZNM1` | 4TB
`SN: Z305S03R` | \ No newline at end of file +| 4TB
`SN: Z305ZNZM` | 4TB
`SN: Z305SPL8` | 4TB
`SN: Z3051AF8` | 4TB
`SN: Z305ZNM1` | 4TB
`SN: Z305S03R` | + +## Related Documentation +- [Related Lab Map Documentation](<../index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/workflows/operations/Hardware Management/ILO/Generic ILO Advanced License Keys.md b/reference/Lab Map/Hardware/iLO License Reference.md similarity index 77% rename from workflows/operations/Hardware Management/ILO/Generic ILO Advanced License Keys.md rename to reference/Lab Map/Hardware/iLO License Reference.md index 9514130..b0dd7e0 100644 --- a/workflows/operations/Hardware Management/ILO/Generic ILO Advanced License Keys.md +++ b/reference/Lab Map/Hardware/iLO License Reference.md @@ -5,6 +5,9 @@ tags: - Licensing --- +## Purpose +Preserve the recorded iLO license reference and its existing usage limitations for lab hardware. + !!! info "Assumptions of Usage" It should go without saying, using one of these keys does not entitle you to support by Hewlett-Packard Enterprise. These are meant for homelab environments where licensing / auditing does not matter. @@ -17,5 +20,8 @@ tags: !!! warning "Do not Use in Production Work Environments" In (rare) cases, these keys can be used as a temporary solution when working in a work environment, then promptly removed after the work is performed. Leaving them installed on a server could lead to legal consequences if Hewlett-Packard Enterprise asked for it while providing support, and it was using one of these keys, it could fail a software licensing audit. - - `REMOVE THE KEY AFTER USAGE` \ No newline at end of file + + `REMOVE THE KEY AFTER USAGE` + +## Related Documentation +- [Related Lab Map Documentation](<../index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/infrastructure/networking/IP Tables/Homelab Server Inventory.md b/reference/Lab Map/Homelab Server Inventory.md similarity index 86% rename from reference/infrastructure/networking/IP Tables/Homelab Server Inventory.md rename to reference/Lab Map/Homelab Server Inventory.md index ac03db3..e41d96c 100644 --- a/reference/infrastructure/networking/IP Tables/Homelab Server Inventory.md +++ b/reference/Lab Map/Homelab Server Inventory.md @@ -5,13 +5,19 @@ tags: - Docker --- +## Purpose +Use this reference for homelab server inventory and the environment-specific values recorded below. + +!!! info "Inventory Reconciliation" + This reference preserves the recorded host and group assignments. Some deployment examples use different hostnames or addresses; reconcile those differences with the running environment before changing inventory or applying a procedure. + ## Overview All servers (physical and virtual) are documented within this specific page. They are written in a specific annotated manner in order to make them copy/paste ready for the Ansible AWX Operator server that interacts with devices in the homelab over `SSH` and `WinRM` protocols. This allows me to automate functions such as updates across the entire homelab declaratively versus individually. -**Note**: This list does not include Docker/Kubernetes-based workloads/servers. Those can be found within the [Container Network IP Table](../../networking/ip-tables/192-168-5-0-container-network.md) document. Given that Ansible does not interact with containers in my homelab (*yet*), these devices are not listed within this document. +**Note**: This list does not include Docker/Kubernetes-based workloads/servers. Those can be found within the [Container Network IP Table]() document. Given that Ansible does not interact with containers in my homelab (*yet*), these devices are not listed within this document. ## Updating Ansible Inventory -Whenever changes are made here, they need to be replicated to the production Ansible AWX Inventory File. This ensures that Ansible AWX is always up-to-date. Simply copy/paste the codeblock below into the linked inventory file, and commit the change with a comment explaining what was added/removed from the inventory list. +Whenever changes are made here, they need to be replicated to the production Ansible AWX Inventory File. This ensures that Ansible AWX is always up-to-date. Simply copy/paste the codeblock below into the linked inventory file, and commit the change with a comment explaining what was added/removed from the inventory list. [:material-ansible: Edit Ansible AWX Inventory File](https://git.bunny-lab.io/GitOps/awx.bunny-lab.io/_edit/main/inventories/homelab.ini){ .md-button } @@ -173,7 +179,7 @@ ansible_connection=ssh 7. Immich Server @ `192.168.3.7` | [Documentation](https://immich.app/docs/install/docker-compose/) 8. Zensical Documentation Server @ `192.168.3.8` | [Documentation](https://hub.docker.com/r/zensical/zensical) 9. FOG Project @ `192.168.3.9` | [Documentation](https://fogproject.org/) -10. Ansible AWX @ `192.168.3.10` | [Documentation](../../../../deployments/automation/ansible/awx/deployment/awx-operator.md) +10. Ansible AWX @ `192.168.3.10` | [Documentation](<../../deployments/automation/AWX/AWX Operator.md>) 11. Minecraft - All The Mods 9 @ `192.168.3.11` | [Documentation](https://www.curseforge.com/minecraft/modpacks/all-the-mods-9) 12. Windows DHCPS Server @ `192.168.3.12` 13. Windows DHCPS Server @ `192.168.3.13` @@ -181,18 +187,18 @@ ansible_connection=ssh 15. Proxmox Mail Gateway @ `192.168.3.15` | [Documentation](https://example.com) 16. Not Currently In-Use @ `192.168.3.16` | [Documentation](https://example.com) 17. Traefik Reverse Proxy @ `192.168.3.17` | [Documentation](https://example.com) -18. Keycloak Server @ `192.168.3.18` | [Documentation](../../../../deployments/services/authentication/keycloak/deployment.md) -19. Docker Container Environment (Portainer) @ `192.168.3.19` | [Documentation](../../../../deployments/platforms/containerization/docker/deploy-portainer.md) -20. PrivacyIDEA @ `192.168.3.20` | [Documentation](../../../../deployments/services/authentication/privacyidea.md) -21. Puppet Server @ `192.168.3.21` | [Documentation](../../../../deployments/automation/puppet/deployment/puppet.md) +18. Keycloak Server @ `192.168.3.18` | [Documentation](<../../deployments/Identity and Certificates/Keycloak/Deploy Keycloak.md>) +19. Docker Container Environment (Portainer) @ `192.168.3.19` | [Documentation](<../../deployments/Containers/Docker/Deploy Portainer.md>) +20. PrivacyIDEA @ `192.168.3.20` | [Documentation](<../../deployments/Identity and Certificates/Privacyidea.md>) +21. Puppet Server @ `192.168.3.21` | [Documentation](<../../deployments/automation/Puppet/Puppet.md>) 22. Not Currently In-Use @ `192.168.3.22` | [Documentation](https://example.com) -23. Hyper-V Failover Cluster @ `192.168.3.23` | [Documentation](../../../../deployments/platforms/virtualization/hyper-v/failover-cluster/deploy-failover-cluster-node.md) +23. Hyper-V Failover Cluster @ `192.168.3.23` | [Documentation](<../../deployments/Virtualization and Storage/Hyper-V/Failover Cluster/Deploy Failover Cluster Node.md>) 24. TrueNAS SCALE @ `192.168.3.24` | [Documentation](https://www.truenas.com/truenas-scale/) 25. Primary Domain Controller @ `192.168.3.25` | [Documentation](https://example.com) 26. Secondary Domain Controller @ `192.168.3.26` | [Documentation](https://example.com) 27. Blue Iris Surveillance @ `192.168.3.27` | [Documentation](https://blueirissoftware.com/) -28. ARK: Survival Ascended Server @ `192.168.3.28` | [Documentation](../../../../deployments/services/gaming/ark-survival-ascended.md) -29. Nextcloud AIO @ `192.168.3.29` | [Documentation](../../../../deployments/services/productivity/nextcloud-aio.md) +28. ARK: Survival Ascended Server @ `192.168.3.28` | [Documentation](<../../deployments/Applications/Gaming and Media/Ark Survival Ascended.md>) +29. Nextcloud AIO @ `192.168.3.29` | [Documentation](<../../deployments/Applications/Files and Collaboration/Nextcloud AIO.md>) 30. Dev-Testing Win11 Lab Environment @ `192.168.3.35` | [Documentation](https://example.com) 31. Windows 11 Work VM @ `192.168.3.31` | [Documentation](https://example.com) 32. Matrix Synapse HomeServer @ `192.168.3.32` | [Documentation](https://github.com/matrix-org/synapse) @@ -239,7 +245,10 @@ ansible_connection=ssh 73. ProxmoxVE Headless Laptop Virtualization Node 74. Rancher Harvester Node 75. Rancher Harvester Cluster VIP -251. Borealis Engine Node 02 -252. Borealis Engine Node 01 -253. Fedora Workstation 42 VM -254. Core Infrastructure Switch 01 (Zyxel GS1920-48) +251. Borealis Engine Node 02 +252. Borealis Engine Node 01 +253. Fedora Workstation 42 VM +254. Core Infrastructure Switch 01 (Zyxel GS1920-48) + +## Related Documentation +- [Related Lab Map Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/infrastructure/networking/IP Tables/172.16.16.0 Sophos Network.md b/reference/Lab Map/Network Address Plans/172.16.16.0 Sophos Network.md similarity index 58% rename from reference/infrastructure/networking/IP Tables/172.16.16.0 Sophos Network.md rename to reference/Lab Map/Network Address Plans/172.16.16.0 Sophos Network.md index cf9a88b..bae4202 100644 --- a/reference/infrastructure/networking/IP Tables/172.16.16.0 Sophos Network.md +++ b/reference/Lab Map/Network Address Plans/172.16.16.0 Sophos Network.md @@ -5,8 +5,11 @@ tags: - Networking --- +## Purpose +Use this reference for 172.16.16.0 sophos network and the environment-specific values recorded below. + ### IP Addresses -Documented IP addresses of Hyper-V Failover Cluster VMs that exist behind the Sophos XG Firewall VM. All of these machines are funneled through the Sophos XG Firewall VM before they are allowed to communicate on the physical network with other devices. +Documented IP addresses of Hyper-V Failover Cluster VMs that exist behind the Sophos XG Firewall VM. All of these machines are funneled through the Sophos XG Firewall VM before they are allowed to communicate on the physical network with other devices. ## 172.16.16.0/24 Network | **IP Address** | **FQDN** | **Additional Notes** | @@ -14,4 +17,7 @@ Documented IP addresses of Hyper-V Failover Cluster VMs that exist behind the So | 172.16.16.1 | LAB-SOPHOS-01.bunny-lab.io | Sophos XG Firewall | | 172.16.16.2 | LAB-IRIS-01 | Blue Iris Surveillance | | 172.16.16.3 | `NOT IN USE` | `NOT IN USE` | -| 172.16.16.4 | `NOT IN USE` | `NOT IN USE` | \ No newline at end of file +| 172.16.16.4 | `NOT IN USE` | `NOT IN USE` | + +## Related Documentation +- [Related Lab Map Documentation](<../index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/infrastructure/networking/IP Tables/192.168.5.0 Container Network.md b/reference/Lab Map/Network Address Plans/192.168.5.0 Container Network.md similarity index 83% rename from reference/infrastructure/networking/IP Tables/192.168.5.0 Container Network.md rename to reference/Lab Map/Network Address Plans/192.168.5.0 Container Network.md index caeca77..490cee3 100644 --- a/reference/infrastructure/networking/IP Tables/192.168.5.0 Container Network.md +++ b/reference/Lab Map/Network Address Plans/192.168.5.0 Container Network.md @@ -5,6 +5,12 @@ tags: - Networking --- +## Purpose +Use this reference for 192.168.5.0 container network and the environment-specific values recorded below. + +!!! info "Address Ownership" + Entries marked undocumented or unknown remain unresolved. Deployment examples elsewhere describe some of the same addresses, but they do not establish current ownership. Confirm the running service before updating this plan. + ### IP Addresses Documented IP addresses of containers. @@ -52,7 +58,7 @@ Documented IP addresses of containers. | 192.168.5.39 | speedtest.bunny-lab.io | Speedtest Tracker Database | | 192.168.5.40 | apprise.bunny-lab.io | Apprise Notification Relaying Service | | 192.168.5.41 | joplin.bunny-lab.io | Joplin Documentation | -| 192.168.5.42 | todo.bunny-lab.io | Tududi Server | +| 192.168.5.42 | todo.bunny-lab.io | Tududi Server | | 192.168.5.43 | `UNDOCUMENTED - Active` | | | 192.168.5.44 | `UNDOCUMENTED - Active` | | | 192.168.5.45 | `UNDOCUMENTED - Active` | | @@ -66,3 +72,6 @@ Documented IP addresses of containers. | 192.168.5.53 | Kavita Server | | | 192.168.3.54 | `UNDOCUMENTED - Active` | | | 192.168.3.55 | Linkding Server | | + +## Related Documentation +- [Related Lab Map Documentation](<../index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/workflows/operations/Power and UPS/APC Cell Wiring Diagram.md b/reference/Lab Map/Power/APC Battery Cell Wiring.md similarity index 68% rename from workflows/operations/Power and UPS/APC Cell Wiring Diagram.md rename to reference/Lab Map/Power/APC Battery Cell Wiring.md index 50cc2d0..758b928 100644 --- a/workflows/operations/Power and UPS/APC Cell Wiring Diagram.md +++ b/reference/Lab Map/Power/APC Battery Cell Wiring.md @@ -5,13 +5,14 @@ tags: - Power --- -**Purpose**: When an APC battery backup's battery dies, you can manually replace the cells and 'refurbish' the battery. The following diagram is how you rewire the cells. +## Purpose +When an APC battery backup's battery dies, you can manually replace the cells and 'refurbish' the battery. The following diagram is how you rewire the cells. !!! warning "Work in Progress" - This document is still being written + This document is still being written ## Wiring Diagram -``` mermaid +```mermaid graph TB %% Define cells and connections Cell1["Cell 1
Black (Negative) to Black (Negative)"] -.-> AndersonNeg["Anderson Connector Negative
(Black)"] @@ -38,10 +39,13 @@ graph TB class AndersonNeg negative; %% Define line colors for clarity - linkStyle 0 stroke:#000,stroke-width:2px; - linkStyle 1 stroke:#ff0000,stroke-width:2px; - linkStyle 2 stroke:#ff0000,stroke-width:2px; - linkStyle 3 stroke:#ff0000,stroke-width:2px; - linkStyle 4 stroke:#ff0000,stroke-width:2px; + linkStyle 0 stroke:#000,stroke-width:2px; + linkStyle 1 stroke:#ff0000,stroke-width:2px; + linkStyle 2 stroke:#ff0000,stroke-width:2px; + linkStyle 3 stroke:#ff0000,stroke-width:2px; + linkStyle 4 stroke:#ff0000,stroke-width:2px; linkStyle 5 stroke:#ff0000,stroke-width:2px; -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Lab Map Documentation](<../index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/workflows/operations/Power and UPS/Battery Backup Power Distribution.md b/reference/Lab Map/Power/UPS Power Distribution.md similarity index 82% rename from workflows/operations/Power and UPS/Battery Backup Power Distribution.md rename to reference/Lab Map/Power/UPS Power Distribution.md index 8b4cefb..dd6cd22 100644 --- a/workflows/operations/Power and UPS/Battery Backup Power Distribution.md +++ b/reference/Lab Map/Power/UPS Power Distribution.md @@ -5,9 +5,15 @@ tags: - Power --- +## Purpose +Record the devices connected to each lab UPS, their estimated runtime, and their shutdown thresholds. Use this reference when planning maintenance or checking power dependencies. + | **Battery Backup** | **Status** | **Connected Device(s)** | **Estimated Runtime** | **Shutdown Threshold** | **UPS Web Management** | | :--- | :--- | :--- | :--- | :--- | :---: | | Outer-Left `#1` | ![](https://status.bunny-lab.io/api/v1/endpoints/battery-backups_outer-left-1-(virt-node-01--10-port-10gbe-network-switch--pfsense-firewall)/uptimes/7d/badge.svg) | - VIRT-NODE-01
- 10-Port 10GbE Network Switch
- pfSense Firewall | 10 Minutes | 3 Minutes Remaining | [:fontawesome-solid-car-battery: Manage](http://192.168.3.4:3052){ .md-button } | | Inner-Left `#2` | ![](https://status.bunny-lab.io/api/v1/endpoints/battery-backups_inner-left-2-(bunny-node-02--24-port-1gbe-network-switch)/uptimes/7d/badge.svg) | - BUNNY-NODE-02
- 24-Port 1GbE Network Switch | 12 Minutes | 3 Minutes Remaining | [:fontawesome-solid-car-battery: Manage](http://192.168.3.5:3052){ .md-button } | | Inner-Right `#3` | ![](https://status.bunny-lab.io/api/v1/endpoints/battery-backups_inner-right-3-(moon-storage-01--wireless-ap)/uptimes/7d/badge.svg) | - MOON-STORAGE-01
- Wireless AP | 16 Minutes | 3 Minutes Remaining | [:fontawesome-solid-car-battery: Manage](http://192.168.3.3:3052){ .md-button } | -| Outer-Right `#4` | ![](https://status.bunny-lab.io/api/v1/endpoints/battery-backups_outer-right-4-(lab-draas-01--lab-pool-01--8-port-1gbe-network-switch--internet-modem--poe-surveillance-cameras)/uptimes/7d/badge.svg) | - LAB-DRAAS-01
- LAB-POOL-01
- 8-Port 1GbE Network Switch
- Internet Modem
- PoE Surveillance Cameras | 13 Minutes | 3 Minutes Remaining | [:fontawesome-solid-car-battery: Manage](http://192.168.3.33:3052){ .md-button } | \ No newline at end of file +| Outer-Right `#4` | ![](https://status.bunny-lab.io/api/v1/endpoints/battery-backups_outer-right-4-(lab-draas-01--lab-pool-01--8-port-1gbe-network-switch--internet-modem--poe-surveillance-cameras)/uptimes/7d/badge.svg) | - LAB-DRAAS-01
- LAB-POOL-01
- 8-Port 1GbE Network Switch
- Internet Modem
- PoE Surveillance Cameras | 13 Minutes | 3 Minutes Remaining | [:fontawesome-solid-car-battery: Manage](http://192.168.3.33:3052){ .md-button } | + +## Related Documentation +- [Related Lab Map Documentation](<../index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/Lab Map/index.md b/reference/Lab Map/index.md new file mode 100644 index 0000000..9f2468c --- /dev/null +++ b/reference/Lab Map/index.md @@ -0,0 +1,23 @@ +--- +tags: + - Lab Map + - Reference + - Documentation +--- + +# Lab Map +## Purpose +Identify where a workload runs, which address plan describes it, and which hardware or power reference applies before following a deployment or repair procedure. + +## Includes +- Find recorded host addresses, inventory groups, and service roles. +- Match container addresses to the services recorded for that subnet. +- Check the separate Sophos network before using its VPN procedures. + +## Find the Right Document +- [Server Inventory]() — Find recorded host addresses, inventory groups, and service roles. +- [Container Address Plan]() — Match container addresses to the services recorded for that subnet. +- [Sophos Address Plan]() — Check the separate Sophos network before using its VPN procedures. +- [UPS Power Distribution]() — Identify the recorded devices, runtimes, and shutdown thresholds for each UPS. +- [Storage Node 01 Disk Layout]() — Identify the physical drive before following the drive-replacement workflow. +- [Replace a TrueNAS Drive](<../../workflows/Virtualization and Storage/TrueNAS/Replace a Drive in Storage Node 01.md>) — Use the physical disk inventory during replacement and update it afterward. diff --git a/deployments/services/DNS/Windows Server/Best Practices.md b/reference/Networking and Access/DNS/Windows DNS Configuration Notes.md similarity index 68% rename from deployments/services/DNS/Windows Server/Best Practices.md rename to reference/Networking and Access/DNS/Windows DNS Configuration Notes.md index eb438c7..6281e3e 100644 --- a/deployments/services/DNS/Windows Server/Best Practices.md +++ b/reference/Networking and Access/DNS/Windows DNS Configuration Notes.md @@ -6,25 +6,25 @@ tags: --- ## Purpose -This document outlines best practices for DNS server configuration in Active Directory environments, focusing on both performance and security considerations. The goal is to enhance the stability, efficiency, and security of DNS infrastructure within enterprise networks. +This document outlines best practices for DNS server configuration in Active Directory environments, focusing on both performance and security considerations. The goal is to enhance the stability, efficiency, and security of DNS infrastructure within enterprise networks. ## Performance Best Practices !!! note "Performance Recommendations Overview" The following list is organized in order of priority, with the most critical practices listed first. ### Redundancy and High Availability -* **Always have at least two DNS servers, preferably three (1 master, 2 slaves).** +- **Always have at least two DNS servers, preferably three (1 master, 2 slaves).** Ensures redundancy and high availability. ### Internal DNS Usage -* **Domain-joined computers should only use internal DNS servers.** +- **Domain-joined computers should only use internal DNS servers.** This ensures that end-user computers can always resolve internal resources and simplifies troubleshooting and management. -* **Extended Reason:** Using only internal DNS servers increases security and streamlines DNS operations. +- **Extended Reason:** Using only internal DNS servers increases security and streamlines DNS operations. ### DNS Server Self-Referencing -* **A DNS server should have `127.0.0.1` loopback as a secondary or tertiary DNS server.** +- **A DNS server should have `127.0.0.1` loopback as a secondary or tertiary DNS server.** Improves the DNS server’s own performance and availability. -* **Extended Reason:** Setting the loopback address as the primary DNS can prevent Active Directory from locating replication partners. Use as secondary or tertiary only. +- **Extended Reason:** Setting the loopback address as the primary DNS can prevent Active Directory from locating replication partners. Use as secondary or tertiary only. !!! info "Recent Changes" The usage of `127.0.0.1` has been changed to pointing to the actual full IP address of the server itself. I need to research this more to determine where this updated guideline came from. For example, if the DNS server IP was `192.168.3.25` you would set that as the value for the secondary DNS server. @@ -33,15 +33,15 @@ This document outlines best practices for DNS server configuration in Active Dir When you are setting up domain controllers / DNS servers, you do not want to use the DC itself as the primary. This can cause all sorts of unexpected issues with reliability and replication. Always have another DNS server as the primary, THEN set the 127.0.0.1 localhost as secondary or tertiary. ### DNS Server Prioritization -* **Prioritize DNS servers based on proximity to endpoints.** +- **Prioritize DNS servers based on proximity to endpoints.** Assign the primary DNS server as the local server, and secondary as a remote branch server, to improve lookup speeds. ### DNS Record Aging and Scavenging -* **Enable DNS record aging/scavenging (preferably 7 days).** +- **Enable DNS record aging/scavenging (preferably 7 days).** Keeps DNS recordsets manageable, which improves lookup performance and troubleshooting. ### Use of CNAME Records -* **Use CNAME records for DNS aliasing. Avoid A records for aliases.** +- **Use CNAME records for DNS aliasing. Avoid A records for aliases.** Updating one host record updates all associated aliases, and PTR records remain properly configured. ## Security Best Practices @@ -49,47 +49,50 @@ This document outlines best practices for DNS server configuration in Active Dir The following list is organized in order of priority, with the most critical practices listed first. ### Network Exposure -* **DNS servers should never be publicly accessible from the internet.** +- **DNS servers should never be publicly accessible from the internet.** This prevents attackers from performing reconnaissance or planning attacks using exposed DNS infrastructure. ### Administrative Access -* **Restrict RDP/remote desktop access to DNS servers/domain controllers to a limited list of administrators.** +- **Restrict RDP/remote desktop access to DNS servers/domain controllers to a limited list of administrators.** Reduces the risk of reconnaissance, reverse shell attacks, and malware installation. ### Use of Slave DNS Servers -* **End-users should be issued only replicated/slave DNS servers.** +- **End-users should be issued only replicated/slave DNS servers.** Protects the master/authoritative DNS server from being directly exposed as an attack vector. -* **Extended Reason:** In branch office scenarios, assign the local replicated server as primary, and main office replicated servers as secondary and tertiary, keeping the master server isolated. +- **Extended Reason:** In branch office scenarios, assign the local replicated server as primary, and main office replicated servers as secondary and tertiary, keeping the master server isolated. ### DNS Server Cache Lockdown -* **Lock the DNS server cache to 100% (read-only).** +- **Lock the DNS server cache to 100% (read-only).** Prevents DNS cache poisoning by allowing cache changes only after TTL expiry. ### DNS Logging -* **Enable DNS logging.** +- **Enable DNS logging.** Facilitates troubleshooting and administration. ### DNS Security Filtering -* **Enable DNS security filtering via DNS forwarder or a security appliance.** +- **Enable DNS security filtering via DNS forwarder or a security appliance.** Use secure public DNS (e.g., 9.9.9.9) or a firewall appliance (e.g., Sophos XG Firewall) to add a security layer to all DNS queries. ### Enable DNSSEC -* **Enable DNSSEC (DNS Security Extensions).** +- **Enable DNSSEC (DNS Security Extensions).** Protects against DNS record spoofing and related attacks. ### DNS Socket Port Randomization -* **Enable DNS socket port randomization.** +- **Enable DNS socket port randomization.** Prevents network attacks by making DNS queries originate from unpredictable ports. -* **Note:** Enabled by default on Windows Server 2016 and newer. +- **Note:** Enabled by default on Windows Server 2016 and newer. ## Additional Notes !!! note "Best Practices Analyzer" It is recommended to run the official Windows Server DNS Best Practices Analyzer (BPA) on your managed servers for insights specific to your domain environment. ## Sources / References -* [Active Directory Pro: DNS Best Practices](https://activedirectorypro.com/dns-best-practices/) -* [Spiceworks: DNS Server Best Practice](https://community.spiceworks.com/topic/1110865-best-practice-for-dns-servers) -* [Microsoft Docs: Creating a DNS Infrastructure Design](https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/plan/creating-a-dns-infrastructure-design) -* [PhoenixNAP: DNS Best Practices Security](https://phoenixnap.com/kb/dns-best-practices-security) -* [Monitis: Best Practices for Active Directory Integrated DNS](https://www.monitis.com/blog/best-practices-for-active-directory-integrated-dns) -* [DNS Knowledge: Authoritative Name Server](https://www.dnsknowledge.com/whatis/authoritative-name-server/) +- [Active Directory Pro: DNS Best Practices](https://activedirectorypro.com/dns-best-practices/) +- [Spiceworks: DNS Server Best Practice](https://community.spiceworks.com/topic/1110865-best-practice-for-dns-servers) +- [Microsoft Docs: Creating a DNS Infrastructure Design](https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/plan/creating-a-dns-infrastructure-design) +- [PhoenixNAP: DNS Best Practices Security](https://phoenixnap.com/kb/dns-best-practices-security) +- [Monitis: Best Practices for Active Directory Integrated DNS](https://www.monitis.com/blog/best-practices-for-active-directory-integrated-dns) +- [DNS Knowledge: Authoritative Name Server](https://www.dnsknowledge.com/whatis/authoritative-name-server/) + +## Related Documentation +- [Related Networking and Access Documentation](<../index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/Networking and Access/index.md b/reference/Networking and Access/index.md new file mode 100644 index 0000000..a0a3d8c --- /dev/null +++ b/reference/Networking and Access/index.md @@ -0,0 +1,23 @@ +--- +tags: + - Networking and Access + - Reference + - Documentation +--- + +# Networking and Access +## Purpose +Find the DNS, proxy, VPN, and remote-access instructions that connect users and services. Use the address plans to identify the intended network before changing connectivity. + +## Includes +- Identify the container network before following service examples. +- Review the documented DNS configuration and its environment assumptions. +- Connect application routes and dynamic configuration delivery. + +## Find the Right Document +- [Network Address Plans](<../Lab Map/Network Address Plans/192.168.5.0 Container Network.md>) — Identify the container network before following service examples. +- [Windows DNS Notes]() — Review the documented DNS configuration and its environment assumptions. +- [Traefik](<../../deployments/Networking and Access/Reverse Proxies/Traefik.md>) — Connect application routes and dynamic configuration delivery. +- [Sophos Site-to-Site VPN](<../../deployments/Networking and Access/Sophos/Configure a Site-to-Site IPsec VPN.md>) — Build the documented tunnel, then use the related reset workflow if needed. +- [Guacamole](<../../deployments/Networking and Access/Remote Access/Apache Guacamole.md>) — Find the browser-based remote-access deployment and its target prerequisites. +- [UniFi Deployment Choices](<../../deployments/Networking and Access/UniFi/Deploy UniFi Controller with Docker.md>) — Start with the Docker example or follow its link to the Ubuntu installation notes. diff --git a/reference/Virtualization and Storage/Proxmox/index.md b/reference/Virtualization and Storage/Proxmox/index.md new file mode 100644 index 0000000..21c8a5f --- /dev/null +++ b/reference/Virtualization and Storage/Proxmox/index.md @@ -0,0 +1,39 @@ +--- +tags: + - Proxmox + - iSCSI + - Storage +--- + +# Proxmox +## Purpose +Choose the Proxmox host, storage, and guest procedure that matches the environment you are operating. Shared LVM over iSCSI and the separate ZFS-over-iSCSI integration are documented alternatives with different management requirements. + +## Includes +- Host and template deployment +- Storage design and capacity changes +- Migration, maintenance, and recovery + +## Choose the Storage Design +- [Shared LVM over iSCSI](<../../../deployments/Virtualization and Storage/Proxmox/Configuring ISCSI Based Cluster Storage.md>) — Use the guide for the documented TrueNAS zvol and shared LVM cluster design. +- [ZFS over iSCSI Integration](<../../../deployments/Virtualization and Storage/Proxmox/ZFS over ISCSI.md>) — Use the separate integration guide when the environment uses its plugin and ZFS management model. + +## Build Hosts and Guests +- [Deploy Proxmox VE](<../../../deployments/Virtualization and Storage/Proxmox/Deploy Proxmox VE.md>) — Review the recorded host version and network layout before applying the examples. +- [Create an Ubuntu Template](<../../../deployments/Virtualization and Storage/Proxmox/Create an Ubuntu Cloud-Init Template.md>) — Review the remaining incomplete configuration before using the template. +- [Install the Guest Agent](<../../../deployments/Virtualization and Storage/Guests/Install the QEMU Guest Agent.md>) — Prepare supported Linux guests for hypervisor integration. + +## Expand Storage +The storage appliance, hypervisor disk, guest partition, and guest filesystem are separate layers. Identify the layer that needs capacity and use the procedure for that layout. + +- [Expand a Linux Guest Filesystem](<../../../workflows/Virtualization and Storage/Linux/Expand a Linux Guest Filesystem.md>) — Continue inside a guest after its virtual disk has grown. +- [Expand ZFS on an iSCSI Client](<../../../workflows/Virtualization and Storage/Linux/Expand an iSCSI-Backed ZFS Filesystem.md>) — Use the workflow for a Linux client that consumes the iSCSI disk directly. +- [Expand a Windows OS Volume](<../../../workflows/Windows and Linux/Windows/Delete Windows Recovery Partition.md>) — Review the documented recovery-partition obstruction and its destructive boundary. + +## Maintain and Recover +- [Upgrade Proxmox VE](<../../../workflows/Virtualization and Storage/Proxmox/Upgrade Proxmox VE from 8 to 9.md>) — Use the recorded release-transition procedure and readiness check. +- [Repair a Migrated Rocky Linux Guest](<../../../workflows/Virtualization and Storage/Proxmox/Repair Rocky Linux After a Veeam Migration.md>) — Follow the boot and network repair notes after a Veeam migration. +- [Repair iSCSI After Reboot](<../../../workflows/Virtualization and Storage/Proxmox/Repair iSCSI Connections After Reboot.md>) — Find the documented reconnection repair. +- [Activate a Missing Volume Group](<../../../workflows/Virtualization and Storage/Proxmox/Manually Activate a Volume Group.md>) — Use the existing LVM recovery commands for the matching layout. +- [Audit Orphaned VM Disks](<../../../workflows/Virtualization and Storage/Proxmox/Detect and Remove Orphaned VM Disks.md>) — Complete the reference checks before removing any volume. +- [Hardware and Power Map](<../../Lab Map/index.md>) — Locate the corresponding disks and UPS dependencies. diff --git a/reference/Virtualization and Storage/index.md b/reference/Virtualization and Storage/index.md new file mode 100644 index 0000000..0b24cc3 --- /dev/null +++ b/reference/Virtualization and Storage/index.md @@ -0,0 +1,23 @@ +--- +tags: + - Virtualization and Storage + - Reference + - Documentation +--- + +# Virtualization and Storage +## Purpose +Follow the relationship between hypervisors, shared storage, guest disks, and recovery procedures. Select the documented storage design before choosing a maintenance command. + +## Includes +- Choose a storage design and find host, guest, migration, and repair procedures. +- Follow the Windows clustering deployment. +- Configure the documented non-clustered migration scenario. + +## Find the Right Document +- [Proxmox and Shared Storage]() — Choose a storage design and find host, guest, migration, and repair procedures. +- [Build a Hyper-V Cluster Node](<../../deployments/Virtualization and Storage/Hyper-V/Failover Cluster/Deploy Failover Cluster Node.md>) — Follow the Windows clustering deployment. +- [Hyper-V Live Migration Authentication](<../../workflows/Virtualization and Storage/Hyper-V/Kerberos Enabled VM Migration.md>) — Configure the documented non-clustered migration scenario. +- [Rebuild Hyper-V Cluster Replication](<../../workflows/Virtualization and Storage/Hyper-V/Failover Cluster/Rebuild Failover Cluster Replication.md>) — Recover an individual VM replica between the documented clusters. +- [Physical Lab References](<../Lab Map/index.md>) — Connect storage and hypervisor work to hardware and power records. +- [Backup and Recovery](<../Backup and Recovery/index.md>) — Find backup and replica context before a disruptive operation. diff --git a/scripts/Bash/Time Adjustment.md b/reference/Windows and Linux/Linux Time Commands.md similarity index 57% rename from scripts/Bash/Time Adjustment.md rename to reference/Windows and Linux/Linux Time Commands.md index 0049d57..f788b61 100644 --- a/scripts/Bash/Time Adjustment.md +++ b/reference/Windows and Linux/Linux Time Commands.md @@ -6,14 +6,18 @@ tags: - Linux --- +## Purpose The commands outlined in this short document are meant to be a quick-reference for setting the timezone and date/time of a Linux-based server. -### Set Timezone: +### Set Timezone ```sh sudo timedatectl set-timezone America/Denver ``` -### Set Time & Date +### Set Time and Date ```sh date -s "1 JAN 2025 03:30:00" -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Windows and Linux Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/Windows and Linux/index.md b/reference/Windows and Linux/index.md new file mode 100644 index 0000000..3d23c5e --- /dev/null +++ b/reference/Windows and Linux/index.md @@ -0,0 +1,23 @@ +--- +tags: + - Windows and Linux + - Reference + - Documentation +--- + +# Windows and Linux +## Purpose +Find workstation and server operating-system setup, updates, and repairs. Storage, networking, and identity tasks are linked to their subject guides when they cross operating-system boundaries. + +## Includes +- Follow the recorded workstation configuration and its local disk assumptions. +- Use the operating-system upgrade notes for the recorded release transition. +- Use the deployment script with its documented share and RMM requirements. + +## Find the Right Document +- [Set Up the Fedora Workstation](<../../deployments/Windows and Linux/Fedora/Set Up the Fedora Workstation.md>) — Follow the recorded workstation configuration and its local disk assumptions. +- [Upgrade Fedora](<../../workflows/Windows and Linux/Linux/Fedora Workstation/Upgrading Versions.md>) — Use the operating-system upgrade notes for the recorded release transition. +- [Upgrade Windows 11](<../../scripts/Windows and Linux/Windows/Upgrade Windows 11 from a UNC Path.md>) — Use the deployment script with its documented share and RMM requirements. +- [Recover After a Windows Update](<../../workflows/Windows and Linux/Windows/Uninstall Updates via DISM.md>) — Work against an offline Windows installation when it cannot boot. +- [Expand Guest Storage](<../../workflows/Virtualization and Storage/Linux/Expand a Linux Guest Filesystem.md>) — Continue from the hypervisor disk change to the guest partition and filesystem. +- [Identity and Remote Management](<../Identity and Certificates/index.md>) — Find domain trust, certificates, and WinRM guidance shared by host administration. diff --git a/reference/foundations/Choose a Document Template.md b/reference/foundations/Choose a Document Template.md new file mode 100644 index 0000000..d08e803 --- /dev/null +++ b/reference/foundations/Choose a Document Template.md @@ -0,0 +1,22 @@ +--- +tags: + - Documentation + - Markdown + - Templates +--- + +## Purpose +Choose the canonical template that matches the primary intent of the document you are creating. The styling guideline remains the single source for template content. + +## Choose by Intent +- [Deployment Templates]() — Build a platform or service; use the container template for a Compose-based application. +- [Workflow Templates]() — Perform maintenance, migration, recovery, or troubleshooting. +- [Script Templates]() — Preserve a complete reusable script or a small operational command. +- [Reference and Inventory Template]() — Record concepts, architecture, addresses, mappings, or stable facts. +- [Index and Blog Templates]() — Introduce a subject folder or preserve a dated narrative. + +## Place and Connect the Page +Use [Documentation Organization]() to choose the subject and add links to the relevant guide and prerequisites. + +## Related Documentation +- [Related Foundations Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/foundations/Documentation Organization.md b/reference/foundations/Documentation Organization.md new file mode 100644 index 0000000..1f59623 --- /dev/null +++ b/reference/foundations/Documentation Organization.md @@ -0,0 +1,36 @@ +--- +tags: + - Documentation + - Information Architecture + - Reference +--- + +## Purpose +Keep Bunny Lab documentation discoverable by both its subject and its operational purpose. Topic guides connect the canonical documents across the five documentation roots. + +## Choose the Document Role +| **Root** | **Primary Intent** | +| :--- | :--- | +| `Deployments` | Build or install the system | +| `Workflows` | Maintain, migrate, repair, or recover it | +| `Scripts` | Preserve a reusable operational utility | +| `Reference` | Explain concepts, inventory, architecture, and subject relationships | +| `Blog` | Preserve dated experience and the reasoning behind decisions | + +## Choose the Subject +Use the same subject names across roots: Applications, Automation, Backup and Recovery, Containers, Identity and Certificates, Networking and Access, Virtualization and Storage, and Windows and Linux. Physical inventory and address plans belong in the Lab Map; authoring standards belong in Foundations. + +## Connect the Knowledge +Link to the applicable prerequisite where it becomes necessary. Link a deployment to its independently useful maintenance procedures, and link those procedures back to the relevant environment or deployment. Keep one canonical copy of scripts and connection settings. A topic guide can describe alternative implementations and explain when each applies. + +## Record Applicability +Use a short, specific admonition for incomplete instructions, version constraints, or conflicting environment notes. Mark an approach historical or superseded only when the recorded evidence supports that conclusion. A file move or formatting review does not establish that a procedure was executed successfully. + +## Maintain Filenames and Links +Use descriptive Title Case filenames and lowercase `index.md` for landing pages. Repair source-relative Markdown links whenever either endpoint moves, including the case of filenames and any changed heading anchors. The [path-change reference]() records the former locations for maintaining bookmarks or server-side redirects. + +## Apply the House Style +Follow [Documentation Styling]() for complete Markdown and document-type requirements. + +## Related Documentation +- [Related Foundations Documentation]() — Find the connected deployments, procedures, and references for this subject. diff --git a/reference/foundations/Documentation Path Changes.md b/reference/foundations/Documentation Path Changes.md new file mode 100644 index 0000000..b5c0666 --- /dev/null +++ b/reference/foundations/Documentation Path Changes.md @@ -0,0 +1,209 @@ +--- +tags: + - Documentation + - Migration + - Reference +--- + +## Purpose +Locate a document after the September 2026 subject reorganization. This reference maps former source paths to their canonical destinations for bookmarks, external references, and deployment-side redirect maintenance. + +## Source Path Mapping +The paths below are relative to the documentation root. Internal source links use the new destinations. This table records the migration; it does not configure HTTP redirects on the documentation server. + +| **Former Source Path** | **Current Document** | +| :--- | :--- | +| `deployments/automation/ansible/awx/Deployment/AWX Operator.md` | [AWX Operator](<../../deployments/automation/AWX/AWX Operator.md>) | +| `deployments/automation/ansible/awx/Deployment/AWX in Minikube.md` | [AWX in Minikube](<../../deployments/automation/AWX/AWX in Minikube.md>) | +| `deployments/automation/ansible/awx/Deployment/Upgrading Issues past 2 10 0.md` | [Repair Upgrades Beyond AWX Operator 2.10.0](<../../workflows/Automation/AWX/Repair Upgrades Beyond AWX Operator 2.10.0.md>) | +| `deployments/automation/puppet/deployment/Puppet Bolt.md` | [Puppet Bolt](<../../deployments/automation/Puppet/Puppet Bolt.md>) | +| `deployments/automation/puppet/deployment/puppet.md` | [Puppet](<../../deployments/automation/Puppet/Puppet.md>) | +| `deployments/platforms/containerization/Docker/Custom Containers/Container Development.md` | [Build and Publish a Container Image](<../../workflows/Containers/Docker/Build and Publish a Container Image.md>) | +| `deployments/platforms/containerization/Docker/Custom Containers/Git Repo Updater.md` | [Git Repo Updater](<../../deployments/Containers/Docker/Git Repo Updater.md>) | +| `deployments/platforms/containerization/Docker/Deploy Portainer.md` | [Deploy Portainer](<../../deployments/Containers/Docker/Deploy Portainer.md>) | +| `deployments/platforms/containerization/kubernetes/deployment/k8s.md` | [K8s](<../../deployments/Containers/Kubernetes/K8s.md>) | +| `deployments/platforms/containerization/kubernetes/deployment/Rancher RKE2.md` | [Rancher RKE2](<../../deployments/Containers/Kubernetes/Rancher RKE2.md>) | +| `deployments/platforms/index.md` | [Virtualization and Storage](<../../deployments/Virtualization and Storage/index.md>) | +| `deployments/platforms/virtualization/Hyper V/Failover Cluster/Deploy Failover Cluster Node.md` | [Deploy Failover Cluster Node](<../../deployments/Virtualization and Storage/Hyper-V/Failover Cluster/Deploy Failover Cluster Node.md>) | +| `deployments/platforms/virtualization/OpenStack/Ansible OpenStack.md` | [Ansible OpenStack](<../../deployments/Virtualization and Storage/OpenStack/Ansible OpenStack.md>) | +| `deployments/platforms/virtualization/OpenStack/Canonical OpenStack.md` | [Canonical OpenStack](<../../deployments/Virtualization and Storage/OpenStack/Canonical OpenStack.md>) | +| `deployments/platforms/virtualization/proxmox/Cloud Init Templates/Ubuntu Server.md` | [Create an Ubuntu Cloud-Init Template](<../../deployments/Virtualization and Storage/Proxmox/Create an Ubuntu Cloud-Init Template.md>) | +| `deployments/platforms/virtualization/proxmox/Configuring ISCSI Based Cluster Storage.md` | [Configuring ISCSI Based Cluster Storage](<../../deployments/Virtualization and Storage/Proxmox/Configuring ISCSI Based Cluster Storage.md>) | +| `deployments/platforms/virtualization/proxmox/Detecting and Removing Orphaned VM Disks.md` | [Detect and Remove Orphaned VM Disks](<../../workflows/Virtualization and Storage/Proxmox/Detect and Remove Orphaned VM Disks.md>) | +| `deployments/platforms/virtualization/proxmox/Fixing iSCSI Connections that Drop at Reboot.md` | [Repair iSCSI Connections After Reboot](<../../workflows/Virtualization and Storage/Proxmox/Repair iSCSI Connections After Reboot.md>) | +| `deployments/platforms/virtualization/proxmox/ProxmoxVE.md` | [Deploy Proxmox VE](<../../deployments/Virtualization and Storage/Proxmox/Deploy Proxmox VE.md>) | +| `deployments/platforms/virtualization/proxmox/ZFS over ISCSI.md` | [ZFS over ISCSI](<../../deployments/Virtualization and Storage/Proxmox/ZFS over ISCSI.md>) | +| `deployments/platforms/virtualization/Rancher Harvester/Harvester.md` | [Harvester](<../../deployments/Virtualization and Storage/Rancher Harvester/Harvester.md>) | +| `deployments/services/Asset Management/Homebox.md` | [Homebox](<../../deployments/Applications/Asset Management/Homebox.md>) | +| `deployments/services/Asset Management/Snipe IT.md` | [Snipe IT](<../../deployments/Applications/Asset Management/Snipe IT.md>) | +| `deployments/services/authentication/Active Directory/Certificate Services.md` | [Certificate Services](<../../deployments/Identity and Certificates/Active Directory/Certificate Services.md>) | +| `deployments/services/authentication/Active Directory/Group Policy/Desktop Shortcut to UNC Path.md` | [Create a Desktop Shortcut to a UNC Path](<../../workflows/Identity and Certificates/Active Directory/Create a Desktop Shortcut to a UNC Path.md>) | +| `deployments/services/authentication/Active Directory/LDAP Settings.md` | [LDAP Connection Settings](<../Identity and Certificates/Active Directory/LDAP Connection Settings.md>) | +| `deployments/services/authentication/Active Directory/Restore Domain Trust.md` | [Restore Domain Trust](<../../workflows/Identity and Certificates/Active Directory/Restore Domain Trust.md>) | +| `deployments/services/authentication/authelia.md` | [Authelia](<../../deployments/Identity and Certificates/Authelia.md>) | +| `deployments/services/authentication/authentik.md` | [Authentik](<../../deployments/Identity and Certificates/Authentik.md>) | +| `deployments/services/authentication/keycloak/deployment.md` | [Deploy Keycloak](<../../deployments/Identity and Certificates/Keycloak/Deploy Keycloak.md>) | +| `deployments/services/authentication/keycloak/oauth2/deployment.md` | [Keycloak Integrations](<../Identity and Certificates/Keycloak Integrations.md>) | +| `deployments/services/authentication/keycloak/oauth2/Gitea OAuth2.md` | [Connect Gitea to Keycloak](<../../workflows/Identity and Certificates/Keycloak/Connect Gitea to Keycloak.md>) | +| `deployments/services/authentication/keycloak/oauth2/Portainer OAuth2.md` | [Connect Portainer to Keycloak](<../../workflows/Identity and Certificates/Keycloak/Connect Portainer to Keycloak.md>) | +| `deployments/services/authentication/privacyidea.md` | [Privacyidea](<../../deployments/Identity and Certificates/Privacyidea.md>) | +| `deployments/services/Automation Tools/Activepieces.md` | [Activepieces](<../../deployments/automation/Tools/Activepieces.md>) | +| `deployments/services/Automation Tools/Node Red.md` | [Node Red](<../../deployments/automation/Tools/Node Red.md>) | +| `deployments/services/Automation Tools/Semaphore UI.md` | [Semaphore UI](<../../deployments/automation/Tools/Semaphore UI.md>) | +| `deployments/services/backup/kopia.md` | [Kopia](<../../deployments/Backup and Recovery/Kopia.md>) | +| `deployments/services/communication/niltalk.md` | [Niltalk](<../../deployments/Applications/Communication/Niltalk.md>) | +| `deployments/services/communication/rocketchat/Autotask Regex Replacer.md` | [Configure Autotask Link Replacement](<../../workflows/Applications/Communication/Rocketchat/Configure Autotask Link Replacement.md>) | +| `deployments/services/communication/rocketchat/deployment.md` | [Deploy Rocket.Chat](<../../deployments/Applications/Communication/Rocketchat/Deploy Rocket.Chat.md>) | +| `deployments/services/DNS/AdGuard Home.md` | [AdGuard Home](<../../deployments/Networking and Access/DNS/AdGuard Home.md>) | +| `deployments/services/DNS/Pi Hole.md` | [Pi Hole](<../../deployments/Networking and Access/DNS/Pi Hole.md>) | +| `deployments/services/DNS/Windows Server/Best Practices.md` | [Windows DNS Configuration Notes](<../Networking and Access/DNS/Windows DNS Configuration Notes.md>) | +| `deployments/services/dashboards/dashy.md` | [Dashy](<../../deployments/Applications/Dashboards/Dashy.md>) | +| `deployments/services/dashboards/Homepage Docker.md` | [Homepage Docker](<../../deployments/Applications/Dashboards/Homepage Docker.md>) | +| `deployments/services/devops/gitea.md` | [Gitea](<../../deployments/automation/Gitea/Gitea.md>) | +| `deployments/services/documentation/docusaurus.md` | [Docusaurus](<../../deployments/automation/Documentation/Docusaurus.md>) | +| `deployments/services/documentation/Material MkDocs.md` | [Material MkDocs](<../../deployments/automation/Documentation/Material MkDocs.md>) | +| `deployments/services/documentation/zensical.md` | [Zensical](<../../deployments/automation/Documentation/Zensical.md>) | +| `deployments/services/edge/nginx.md` | [Nginx](<../../deployments/Networking and Access/Reverse Proxies/Nginx.md>) | +| `deployments/services/edge/traefik.md` | [Traefik](<../../deployments/Networking and Access/Reverse Proxies/Traefik.md>) | +| `deployments/services/email/Microsoft Exchange/Configuring ACME Letsencrypt Bot.md` | [Configuring ACME Letsencrypt Bot](<../../deployments/Applications/Email/Microsoft Exchange/Configuring ACME Letsencrypt Bot.md>) | +| `deployments/services/email/Microsoft Exchange/Preparing for Cumulative Updates.md` | [Prepare for Cumulative Updates](<../../workflows/Applications/Email/Microsoft Exchange/Prepare for Cumulative Updates.md>) | +| `deployments/services/email/Proxmox Mail Gateway/Deploying PMG.md` | [Integrate PMG with Mailcow](<../../deployments/Applications/Email/Proxmox Mail Gateway/Integrate PMG with Mailcow.md>) | +| `deployments/services/email/iRedMail/Deploy iRedMail.md` | [Deploy iRedMail](<../../deployments/Applications/Email/iRedMail/Deploy iRedMail.md>) | +| `deployments/services/email/iRedMail/Query SMTP Outgoing Queue.md` | [Inspect the Outgoing SMTP Queue](<../../workflows/Applications/Email/iRedMail/Inspect the Outgoing SMTP Queue.md>) | +| `deployments/services/email/iRedMail/Quick Server Settings.md` | [iRedMail Connection Settings](<../Applications/Email/iRedMail Connection Settings.md>) | +| `deployments/services/email/mailcow.md` | [mailcow](<../../deployments/Applications/Email/mailcow.md>) | +| `deployments/services/File Services/Windows Server/DFS Namespaces with Replication.md` | [DFS Namespaces with Replication](<../../deployments/Applications/Files and Collaboration/Windows Server/DFS Namespaces with Replication.md>) | +| `deployments/services/gaming/Ark Survival Ascended.md` | [Ark Survival Ascended](<../../deployments/Applications/Gaming and Media/Ark Survival Ascended.md>) | +| `deployments/services/gaming/pterodactyl.md` | [Pterodactyl](<../../deployments/Applications/Gaming and Media/Pterodactyl.md>) | +| `deployments/services/gaming/valheim.md` | [Valheim](<../../deployments/Applications/Gaming and Media/Valheim.md>) | +| `deployments/services/Home and IOT/Frigate.md` | [Frigate](<../../deployments/Applications/Home Automation/Frigate.md>) | +| `deployments/services/Home and IOT/HomeAssistant.md` | [HomeAssistant](<../../deployments/Applications/Home Automation/HomeAssistant.md>) | +| `deployments/services/index.md` | [Applications](<../../deployments/Applications/index.md>) | +| `deployments/services/Media and Gaming/Emulatorjs.md` | [Emulatorjs](<../../deployments/Applications/Gaming and Media/Emulatorjs.md>) | +| `deployments/services/Media and Gaming/Pyload.md` | [Pyload](<../../deployments/Applications/Files and Collaboration/Pyload.md>) | +| `deployments/services/Microsoft 365/Change MFA Settings.md` | [Change MFA Settings](<../../workflows/Identity and Certificates/Microsoft 365/Change MFA Settings.md>) | +| `deployments/services/Microsoft 365/Seize Control of Personal OneDrive Data of Another User.md` | [Access Another User OneDrive Data](<../../workflows/Applications/Files and Collaboration/Microsoft 365/Access Another User OneDrive Data.md>) | +| `deployments/services/monitoring/gatus.md` | [Gatus](<../../deployments/Applications/Monitoring/Gatus.md>) | +| `deployments/services/monitoring/Speedtest Tracker.md` | [Speedtest Tracker](<../../deployments/Applications/Monitoring/Speedtest Tracker.md>) | +| `deployments/services/monitoring/uptimekuma.md` | [UptimeKuma](<../../deployments/Applications/Monitoring/UptimeKuma.md>) | +| `deployments/services/notifications/ntfy.md` | [Ntfy](<../../deployments/Applications/Monitoring/Ntfy.md>) | +| `deployments/services/productivity/Collabora Code Server.md` | [Collabora Code Server](<../../deployments/Applications/Files and Collaboration/Collabora Code Server.md>) | +| `deployments/services/productivity/Nextcloud AIO.md` | [Nextcloud AIO](<../../deployments/Applications/Files and Collaboration/Nextcloud AIO.md>) | +| `deployments/services/productivity/nextcloud.md` | [Nextcloud](<../../deployments/Applications/Files and Collaboration/Nextcloud.md>) | +| `deployments/services/productivity/OnlyOffice EE.md` | [OnlyOffice EE](<../../deployments/Applications/Files and Collaboration/OnlyOffice EE.md>) | +| `deployments/services/productivity/Stirling PDF.md` | [Stirling PDF](<../../deployments/Applications/Files and Collaboration/Stirling PDF.md>) | +| `deployments/services/productivity/trilium.md` | [Trilium](<../../deployments/Applications/Files and Collaboration/Trilium.md>) | +| `deployments/services/productivity/wordpress.md` | [Wordpress](<../../deployments/Applications/Files and Collaboration/Wordpress.md>) | +| `deployments/services/rmm/tacticalrmm.md` | [TacticalRMM](<../../deployments/automation/Remote Management/TacticalRMM.md>) | +| `deployments/services/Remote Access/Apache Guacamole.md` | [Apache Guacamole](<../../deployments/Networking and Access/Remote Access/Apache Guacamole.md>) | +| `deployments/services/Remote Access/Firefox.md` | [Firefox](<../../deployments/Networking and Access/Remote Access/Firefox.md>) | +| `deployments/services/Security and Utility/Changedetection.md` | [Changedetection](<../../deployments/Applications/Utilities/Changedetection.md>) | +| `deployments/services/Security and Utility/Cyberchef.md` | [Cyberchef](<../../deployments/Applications/Utilities/Cyberchef.md>) | +| `deployments/services/Security and Utility/IT Tools.md` | [IT Tools](<../../deployments/Applications/Utilities/IT Tools.md>) | +| `deployments/services/Security and Utility/Password Pusher.md` | [Password Pusher](<../../deployments/Identity and Certificates/Password Pusher.md>) | +| `deployments/services/Security and Utility/Searx.md` | [Searx](<../../deployments/Applications/Utilities/Searx.md>) | +| `deployments/services/Security and Utility/Vaultwarden.md` | [Vaultwarden](<../../deployments/Identity and Certificates/Vaultwarden.md>) | +| `deployments/services/cPanel/Creating Email Server.md` | [Create a cPanel Email Server](<../../deployments/Applications/Email/cPanel/Create a cPanel Email Server.md>) | +| `reference/foundations/Templates/Document Template.md` | [Choose a Document Template]() | +| `reference/infrastructure/hardware/Cluster Node 01/Disk Arrays.md` | [Cluster Node 01 Disk Layout](<../Lab Map/Hardware/Cluster Node 01 Disk Layout.md>) | +| `reference/infrastructure/hardware/Cluster Node 02/Disk Arrays.md` | [Cluster Node 02 Disk Layout](<../Lab Map/Hardware/Cluster Node 02 Disk Layout.md>) | +| `reference/infrastructure/hardware/Cluster Node 03/Disk Arrays.md` | [Cluster Node 03 Disk Layout](<../Lab Map/Hardware/Cluster Node 03 Disk Layout.md>) | +| `reference/infrastructure/hardware/index.md` | [Lab Map](<../Lab Map/index.md>) | +| `reference/infrastructure/hardware/Storage Node 01 Truenas Core/Disk Arrays.md` | [Storage Node 01 TrueNAS Core Disk Layout](<../Lab Map/Hardware/Storage Node 01 TrueNAS Core Disk Layout.md>) | +| `reference/infrastructure/hardware/Storage Node 01 Truenas Core/Replacing a Drive.md` | [Replace a Drive in Storage Node 01](<../../workflows/Virtualization and Storage/TrueNAS/Replace a Drive in Storage Node 01.md>) | +| `reference/infrastructure/hardware/Storage Node 02 Truenas Scale/Disk Arrays.md` | [Storage Node 02 TrueNAS Scale Disk Layout](<../Lab Map/Hardware/Storage Node 02 TrueNAS Scale Disk Layout.md>) | +| `reference/infrastructure/networking/Controllers/UniFi Controller.md` | [Deploy UniFi Controller with Docker](<../../deployments/Networking and Access/UniFi/Deploy UniFi Controller with Docker.md>) | +| `reference/infrastructure/networking/Controllers/UniFi Network Server Controller.md` | [Deploy UniFi Network Server on Ubuntu](<../../deployments/Networking and Access/UniFi/Deploy UniFi Network Server on Ubuntu.md>) | +| `reference/infrastructure/networking/Docker Networking/Creating a Macvlan Sub Interface for Docker.md` | [Create a Macvlan Subinterface](<../../workflows/Containers/Docker/Create a Macvlan Subinterface.md>) | +| `reference/infrastructure/networking/Docker Networking/Docker Networking.md` | [Create the Docker Network](<../../deployments/Containers/Docker/Create the Docker Network.md>) | +| `reference/infrastructure/networking/Firewall and Routing/Sophos/Configure LAN Bridging.md` | [Configure LAN Bridging](<../../workflows/Networking and Access/Sophos/Configure LAN Bridging.md>) | +| `reference/infrastructure/networking/Firewall and Routing/Sophos/VPN/SSL VPN/Configuring Remote VPN RDP Access.md` | [Configure RDP Access over SSL VPN](<../../workflows/Networking and Access/Sophos/Configure RDP Access over SSL VPN.md>) | +| `reference/infrastructure/networking/Firewall and Routing/Sophos/VPN/Site to Site VPNs/IPSEC/Automatic Tunnel Resetting.md` | [Automatically Reset an IPsec Tunnel](<../../workflows/Networking and Access/Sophos/Automatically Reset an IPsec Tunnel.md>) | +| `reference/infrastructure/networking/Firewall and Routing/Sophos/VPN/Site to Site VPNs/IPSEC/Tunnel Creation.md` | [Configure a Site-to-Site IPsec VPN](<../../deployments/Networking and Access/Sophos/Configure a Site-to-Site IPsec VPN.md>) | +| `reference/infrastructure/networking/IP Tables/172.16.16.0 Sophos Network.md` | [172.16.16.0 Sophos Network](<../Lab Map/Network Address Plans/172.16.16.0 Sophos Network.md>) | +| `reference/infrastructure/networking/IP Tables/192.168.5.0 Container Network.md` | [192.168.5.0 Container Network](<../Lab Map/Network Address Plans/192.168.5.0 Container Network.md>) | +| `reference/infrastructure/networking/IP Tables/Homelab Server Inventory.md` | [Homelab Server Inventory](<../Lab Map/Homelab Server Inventory.md>) | +| `reference/infrastructure/networking/index.md` | [Networking and Access](<../Networking and Access/index.md>) | +| `reference/infrastructure/networking/Linux Networking/Modifying IP Address of Server.md` | [Change a Server IP Address](<../../workflows/Networking and Access/Linux/Change a Server IP Address.md>) | +| `reference/infrastructure/networking/Misc/Tuya Smart Lights.md` | [Connect Tuya Smart Lights](<../../workflows/Applications/Home Automation/Connect Tuya Smart Lights.md>) | +| `reference/infrastructure/networking/vpn/netbird.md` | [Deploy NetBird on Rocky Linux](<../../deployments/Networking and Access/NetBird/Deploy NetBird on Rocky Linux.md>) | +| `scripts/Bash/Configure SSH Key Authentication.md` | [Configure SSH Key Authentication](<../../workflows/Networking and Access/Linux/Configure SSH Key Authentication.md>) | +| `scripts/Bash/Fix Displaylink Issues on Linux.md` | [Repair DisplayLink USB Authorization](<../../workflows/Windows and Linux/Linux/Repair DisplayLink USB Authorization.md>) | +| `scripts/Bash/Git Repo Updater.md` | [Git Repo Updater](<../../scripts/Automation/Gitea/Git Repo Updater.md>) | +| `scripts/Bash/Install QEMU Guest Agent.md` | [Install the QEMU Guest Agent](<../../deployments/Virtualization and Storage/Guests/Install the QEMU Guest Agent.md>) | +| `scripts/Bash/Install XRDP.md` | [Install XRDP on Ubuntu](<../../deployments/Networking and Access/Remote Access/Install XRDP on Ubuntu.md>) | +| `scripts/Bash/Mdadm Grow Array Size.md` | [Grow an mdadm Array](<../../scripts/Virtualization and Storage/Linux/Grow an mdadm Array.md>) | +| `scripts/Bash/Open Port Checker.md` | [Check Open Ports](<../../scripts/Networking and Access/Check Open Ports.md>) | +| `scripts/Bash/ProxmoxVE/Deeplab Rollback Script.md` | [Deeplab Rollback Script](<../../scripts/Virtualization and Storage/Proxmox/Deeplab Rollback Script.md>) | +| `scripts/Bash/Time Adjustment.md` | [Linux Time Commands](<../Windows and Linux/Linux Time Commands.md>) | +| `scripts/Bash/Transfer Docker Containers.md` | [Transfer Docker Containers Between Hosts](<../../workflows/Containers/Docker/Transfer Docker Containers Between Hosts.md>) | +| `scripts/Bash/Transfer Files with Netcat.md` | [Transfer Files with Netcat](<../../workflows/Applications/Files and Collaboration/Transfer Files with Netcat.md>) | +| `scripts/batch/Blue Iris/Server Watchdog.md` | [Blue Iris Server Watchdog](<../../scripts/Applications/Home Automation/Blue Iris Server Watchdog.md>) | +| `scripts/batch/robocopy.md` | [Robocopy Command Reference](<../Applications/Files and Collaboration/Robocopy Command Reference.md>) | +| `scripts/Powershell/Azure/Check Email Aliases.md` | [Check Email Aliases](<../../scripts/Identity and Certificates/Microsoft 365/Check Email Aliases.md>) | +| `scripts/Powershell/Azure/Connect to Azure AD.md` | [Connect to Azure AD](<../../scripts/Identity and Certificates/Microsoft 365/Connect to Azure AD.md>) | +| `scripts/Powershell/Exchange Online/Connect to Exchange Online.md` | [Connect to Exchange Online](<../../scripts/Applications/Email/Exchange Online/Connect to Exchange Online.md>) | +| `scripts/Powershell/General Purpose/DNS Hierarchy Correction.md` | [Correct DNS Server Priority](<../../scripts/Networking and Access/Correct DNS Server Priority.md>) | +| `scripts/Powershell/General Purpose/Directory Walker.md` | [Directory Walker](<../../scripts/Applications/Files and Collaboration/Directory Walker.md>) | +| `scripts/Powershell/General Purpose/File Finder.md` | [File Finder](<../../scripts/Applications/Files and Collaboration/File Finder.md>) | +| `scripts/Powershell/General Purpose/Fix Corrupted Windows Updates.md` | [Repair Windows Update Components](<../../scripts/Windows and Linux/Windows/Repair Windows Update Components.md>) | +| `scripts/Powershell/General Purpose/Force GPUpdate Domain Wide.md` | [Force Group Policy Updates Across the Domain](<../../scripts/Identity and Certificates/Active Directory/Force Group Policy Updates Across the Domain.md>) | +| `scripts/Powershell/General Purpose/Inactive User Profile Data Cleanup.md` | [Clean Up Inactive User Profiles](<../../scripts/Windows and Linux/Windows/Clean Up Inactive User Profiles.md>) | +| `scripts/Powershell/General Purpose/Rclone.md` | [Rclone Command Reference](<../Applications/Files and Collaboration/Rclone Command Reference.md>) | +| `scripts/Powershell/General Purpose/Remotely Change DNS Records.md` | [Change DNS Client Server Settings Remotely](<../../scripts/Networking and Access/Change DNS Client Server Settings Remotely.md>) | +| `scripts/Powershell/General Purpose/Restart Service Domain Wide.md` | [Start the RMM Agent Service Across the Domain](<../../scripts/Windows and Linux/Windows/Start the RMM Agent Service Across the Domain.md>) | +| `scripts/Powershell/General Purpose/Windows 11 Upgrade via UNC Path.md` | [Upgrade Windows 11 from a UNC Path](<../../scripts/Windows and Linux/Windows/Upgrade Windows 11 from a UNC Path.md>) | +| `scripts/Powershell/Hyper V/Collapse Differencing Disk Chains.md` | [Collapse Differencing Disk Chains](<../../scripts/Virtualization and Storage/Hyper-V/Collapse Differencing Disk Chains.md>) | +| `scripts/Powershell/Hyper V/Delete Locked VHDX File.md` | [Delete Locked VHDX File](<../../scripts/Virtualization and Storage/Hyper-V/Delete Locked VHDX File.md>) | +| `scripts/Powershell/Hyper V/Failover Cluster/Force Reboot Remote Cluster Node.md` | [Force Reboot Remote Cluster Node](<../../scripts/Virtualization and Storage/Hyper-V/Failover Cluster/Force Reboot Remote Cluster Node.md>) | +| `scripts/Powershell/Hyper V/Failover Cluster/Replication Bumper.md` | [Replication Bumper](<../../scripts/Virtualization and Storage/Hyper-V/Failover Cluster/Replication Bumper.md>) | +| `scripts/Powershell/Minecraft Server/Update Script.md` | [Update the ATM10 Minecraft Server](<../../scripts/Applications/Gaming and Media/Update the ATM10 Minecraft Server.md>) | +| `scripts/Powershell/Nextcloud/Upload Data to Nextcloud Share.md` | [Upload Data to a Nextcloud Share](<../../scripts/Applications/Files and Collaboration/Upload Data to a Nextcloud Share.md>) | +| `scripts/Powershell/Reporting/Get Password Expiration.md` | [Get Password Expiration](<../../scripts/Identity and Certificates/Active Directory/Reports/Get Password Expiration.md>) | +| `scripts/Powershell/Reporting/Inactive Computers.md` | [Inactive Computers](<../../scripts/Identity and Certificates/Active Directory/Reports/Inactive Computers.md>) | +| `scripts/Powershell/Reporting/Inactive Users.md` | [Inactive Users](<../../scripts/Identity and Certificates/Active Directory/Reports/Inactive Users.md>) | +| `scripts/Powershell/SMB/Detailed Permission Report All Shares.md` | [Report NTFS Permissions Across Shares](<../../scripts/Applications/Files and Collaboration/SMB/Report NTFS Permissions Across Shares.md>) | +| `scripts/Powershell/SMB/Top Level Permission Report All Shares.md` | [Report SMB Share Permissions](<../../scripts/Applications/Files and Collaboration/SMB/Report SMB Share Permissions.md>) | +| `scripts/Services/Email/Microsoft Exchange/DAG/Database Management.md` | [Manage DAG Database Copies](<../../workflows/Applications/Email/Microsoft Exchange/Manage DAG Database Copies.md>) | +| `scripts/Services/Email/Microsoft Exchange/DAG/Perform Exchange SE DAG Rolling Updates.md` | [Perform Exchange SE DAG Rolling Updates](<../../workflows/Applications/Email/Microsoft Exchange/Perform Exchange SE DAG Rolling Updates.md>) | +| `scripts/Services/Email/Microsoft Exchange/Restart Exchange Services.md` | [Start Exchange Services](<../../scripts/Applications/Email/Microsoft Exchange/Start Exchange Services.md>) | +| `scripts/Services/Email/Microsoft Exchange/Set Mailbox Auto Reply.md` | [Set Mailbox Auto Reply](<../../scripts/Applications/Email/Microsoft Exchange/Set Mailbox Auto Reply.md>) | +| `workflows/operations/automation/ansible/AWX/AWX Kerberos Implementation.md` | [AWX Kerberos Implementation](<../../workflows/Automation/AWX/AWX Kerberos Implementation.md>) | +| `workflows/operations/automation/ansible/AWX/Connect AWX to Gitea.md` | [Connect AWX to Gitea](<../../workflows/Automation/AWX/Connect AWX to Gitea.md>) | +| `workflows/operations/automation/ansible/credentials/Custom Credential Types/WinRM.md` | [Custom Kerberos WinRM Credential](<../Automation/AWX/Custom Kerberos WinRM Credential.md>) | +| `workflows/operations/automation/ansible/credentials/overview.md` | [Credential Configuration Examples](<../Automation/AWX/Credential Configuration Examples.md>) | +| `workflows/operations/automation/ansible/Enable WinRM on Windows Devices.md` | [Enable WinRM over HTTPS](<../../workflows/Identity and Certificates/Windows/Enable WinRM over HTTPS.md>) | +| `workflows/operations/automation/ansible/inventories/overview.md` | [Inventory Structure and Variables](<../Automation/AWX/Inventory Structure and Variables.md>) | +| `workflows/operations/automation/ansible/playbooks/playbooks.md` | [Playbook Catalog](<../Automation/AWX/Playbook Catalog.md>) | +| `workflows/operations/automation/ansible/projects/overview.md` | [Projects and Source Control](<../Automation/AWX/Projects and Source Control.md>) | +| `workflows/operations/automation/ansible/templates/overview.md` | [Job Template Configuration](<../Automation/AWX/Job Template Configuration.md>) | +| `workflows/operations/Backups and DR/Veeam Backup Replication/Backup Agent Takeover.md` | [Backup Agent Takeover](<../../workflows/Backup and Recovery/Veeam/Backup Agent Takeover.md>) | +| `workflows/operations/Backups and DR/Veeam Backup Replication/Core Veeam Concepts.md` | [Veeam Concepts](<../Backup and Recovery/Veeam Concepts.md>) | +| `workflows/operations/Backups and DR/Veeam Backup Replication/Manually Pruning Backups.md` | [Manually Pruning Backups](<../../workflows/Backup and Recovery/Veeam/Manually Pruning Backups.md>) | +| `workflows/operations/Backups and DR/Veeam Backup Replication/Migrating VMs to ProxmoxVE.md` | [Repair Rocky Linux After a Veeam Migration](<../../workflows/Virtualization and Storage/Proxmox/Repair Rocky Linux After a Veeam Migration.md>) | +| `workflows/operations/Backups and DR/Veeam Backup Replication/Migrating VSPC Backup Repositories.md` | [Migrating VSPC Backup Repositories](<../../workflows/Backup and Recovery/Veeam/Migrating VSPC Backup Repositories.md>) | +| `workflows/operations/Backups and DR/Veeam Backup Replication/Retention Best Practices.md` | [Veeam Retention Policy Example](<../Backup and Recovery/Veeam Retention Policy Example.md>) | +| `workflows/operations/Backups and DR/Veeam Backup Replication/Troubleshooting/Failed to Validate Certificates of Some Gateways.md` | [Failed to Validate Certificates of Some Gateways](<../../workflows/Backup and Recovery/Veeam/Failed to Validate Certificates of Some Gateways.md>) | +| `workflows/operations/Hardware Management/ILO/Generic ILO Advanced License Keys.md` | [iLO License Reference](<../Lab Map/Hardware/iLO License Reference.md>) | +| `workflows/operations/Linux/CachyOS/Restrict Monitors on Plasma Login Screen.md` | [Restrict Monitors on Plasma Login Screen](<../../workflows/Windows and Linux/Linux/CachyOS/Restrict Monitors on Plasma Login Screen.md>) | +| `workflows/operations/Linux/Expand ISCSI Based ZFS Filesystem.md` | [Expand an iSCSI-Backed ZFS Filesystem](<../../workflows/Virtualization and Storage/Linux/Expand an iSCSI-Backed ZFS Filesystem.md>) | +| `workflows/operations/Linux/Expanding Linux Filesystems.md` | [Expand a Linux Guest Filesystem](<../../workflows/Virtualization and Storage/Linux/Expand a Linux Guest Filesystem.md>) | +| `workflows/operations/Linux/Fedora Workstation/Full Setup.md` | [Set Up the Fedora Workstation](<../../deployments/Windows and Linux/Fedora/Set Up the Fedora Workstation.md>) | +| `workflows/operations/Linux/Fedora Workstation/Install DE into Fedora Server.md` | [Install XFCE and RustDesk on Fedora Server](<../../deployments/Networking and Access/Remote Access/Install XFCE and RustDesk on Fedora Server.md>) | +| `workflows/operations/Linux/Fedora Workstation/Install Flatpak Apps.md` | [Install Flatpak Apps](<../../workflows/Windows and Linux/Linux/Fedora Workstation/Install Flatpak Apps.md>) | +| `workflows/operations/Linux/Fedora Workstation/Upgrading Versions.md` | [Upgrading Versions](<../../workflows/Windows and Linux/Linux/Fedora Workstation/Upgrading Versions.md>) | +| `workflows/operations/Power and UPS/APC Cell Wiring Diagram.md` | [APC Battery Cell Wiring](<../Lab Map/Power/APC Battery Cell Wiring.md>) | +| `workflows/operations/Power and UPS/Battery Backup Power Distribution.md` | [UPS Power Distribution](<../Lab Map/Power/UPS Power Distribution.md>) | +| `workflows/operations/Windows/Change Windows Edition.md` | [Change Windows Edition](<../../workflows/Windows and Linux/Windows/Change Windows Edition.md>) | +| `workflows/operations/Windows/Delete Windows Recovery Partition.md` | [Delete Windows Recovery Partition](<../../workflows/Windows and Linux/Windows/Delete Windows Recovery Partition.md>) | +| `workflows/operations/Windows/Uninstall Updates via DISM.md` | [Uninstall Updates via DISM](<../../workflows/Windows and Linux/Windows/Uninstall Updates via DISM.md>) | +| `workflows/operations/Windows/VSS/Delete Shadow Copies.md` | [Delete Shadow Copies](<../../workflows/Windows and Linux/Windows/VSS/Delete Shadow Copies.md>) | +| `workflows/operations/Windows/Windows 11/Creating a Local Account on Win11.md` | [Creating a Local Account on Win11](<../../workflows/Windows and Linux/Windows/Windows 11/Creating a Local Account on Win11.md>) | +| `workflows/operations/Windows/Windows Server/SSL Certificates/Convert SSL Certificates into PFX Files.md` | [Convert Certificates to PFX](<../../workflows/Identity and Certificates/Certificates/Convert Certificates to PFX.md>) | +| `workflows/Platforms/Containerization/Kubernetes/Migrating Docker Compose YML to K8s.md` | [Migrating Docker Compose YML to K8s](<../../workflows/Containers/Kubernetes/Migrating Docker Compose YML to K8s.md>) | +| `workflows/Platforms/Virtualization/Hyper V/Failover Cluster/Rebuild Failover Cluster Replication.md` | [Rebuild Failover Cluster Replication](<../../workflows/Virtualization and Storage/Hyper-V/Failover Cluster/Rebuild Failover Cluster Replication.md>) | +| `workflows/Platforms/Virtualization/Hyper V/Forcefully Stop GuestVM.md` | [Forcefully Stop GuestVM](<../../workflows/Virtualization and Storage/Hyper-V/Forcefully Stop GuestVM.md>) | +| `workflows/Platforms/Virtualization/Hyper V/Kerberos Enabled VM Migration.md` | [Kerberos Enabled VM Migration](<../../workflows/Virtualization and Storage/Hyper-V/Kerberos Enabled VM Migration.md>) | +| `workflows/Platforms/Virtualization/Proxmox/Common Tasks.md` | [Remove a Node from a Proxmox Cluster](<../../workflows/Virtualization and Storage/Proxmox/Remove a Node from a Proxmox Cluster.md>) | +| `workflows/Platforms/Virtualization/Proxmox/Operations/Manually Activate Volume Group.md` | [Manually Activate a Volume Group](<../../workflows/Virtualization and Storage/Proxmox/Manually Activate a Volume Group.md>) | +| `workflows/Platforms/Virtualization/Proxmox/Operations/Upgrade PVE from 8 to 9.md` | [Upgrade Proxmox VE from 8 to 9](<../../workflows/Virtualization and Storage/Proxmox/Upgrade Proxmox VE from 8 to 9.md>) | diff --git a/reference/foundations/Templates/Document Template.md b/reference/foundations/Templates/Document Template.md deleted file mode 100644 index d3c3292..0000000 --- a/reference/foundations/Templates/Document Template.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -tags: - - Documentation - - Templates - - Markdown ---- - -**Purpose**: PLACEHOLDER - -## Docker Configuration -```yaml title="docker-compose.yml" -PLACEHOLDER -``` - -```yaml title=".env" -PLACEHOLDER -``` - -## Traefik Reverse Proxy Configuration -If the container does not run on the same host as Traefik, you will need to manually add configuration to Traefik's dynamic config file, outlined below. -```yaml -http: - routers: - PLACEHOLDER: - entryPoints: - - websecure - tls: - certResolver: myresolver - service: PLACEHOLDER - rule: Host(`PLACEHOLDER.bunny-lab.io`) - - services: - PLACEHOLDER: - loadBalancer: - servers: - - url: http://PLACEHOLDER:80 - passHostHeader: true -``` \ No newline at end of file diff --git a/reference/foundations/index.md b/reference/foundations/index.md index 5c5a108..8fe4edb 100644 --- a/reference/foundations/index.md +++ b/reference/foundations/index.md @@ -1,47 +1,21 @@ --- tags: - - Operations - - Index + - Foundations + - Reference - Documentation --- # Foundations ## Purpose -Defines the baseline documentation standards, shared references, and structural conventions used everywhere else in this knowledgebase. +Maintain the knowledgebase using the authoritative styling contract, document roles, and source-relative links. ## Includes -- Documentation styling contract -- Inventory and naming conventions -- Shared templates and glossary references +- Apply the authoritative Markdown and document-type contract. +- Choose the content role, subject, and related guides for a new page. +- Use the canonical template for the intended page type. -## New Document Template -````markdown -# -## Purpose - - -!!! info "Assumptions" - - - - - -## Scope -- -- - -## Procedure -```sh -# Commands go here (grouped and annotated) -``` - -## Validation -- - -## Troubleshooting -### Symptoms -- - -### Resolution -```sh -# Fix steps -``` -```` +## Find the Right Document +- [Documentation Styling]() — Apply the authoritative Markdown and document-type contract. +- [Documentation Organization]() — Choose the content role, subject, and related guides for a new page. +- [Choose a Document Template]() — Use the canonical template for the intended page type. +- [Documentation Path Changes]() — Map former source paths to their new locations when maintaining external links. diff --git a/reference/index.md b/reference/index.md index abed588..742ed65 100644 --- a/reference/index.md +++ b/reference/index.md @@ -1,15 +1,33 @@ --- tags: - Reference - - Index - Documentation --- # Reference ## Purpose -Stable supporting documentation used by deployments and workflows. +Start with a subject guide to connect the lab inventory, deployment instructions, operational procedures, and reusable scripts. ## Includes -- Documentation foundations and templates -- Hardware inventory and storage layouts -- Networking topology and infrastructure references +- Lab Map +- Virtualization and Storage +- Containers +- Networking and Access +- Identity and Certificates +- Automation +- Backup and Recovery +- Applications +- Windows and Linux +- Foundations + +## Start with a Subject +- [Lab Map]() — Identify where a workload runs, which address plan describes it, and which hardware or power reference applies before following a deployment or repair procedure. +- [Virtualization and Storage]() — Follow the relationship between hypervisors, shared storage, guest disks, and recovery procedures. Select the documented storage design before choosing a maintenance command. +- [Containers]() — Prepare the Docker or Kubernetes environment used by application deployments, then follow the operating procedures for building, moving, and exposing workloads. +- [Networking and Access]() — Find the DNS, proxy, VPN, and remote-access instructions that connect users and services. Use the address plans to identify the intended network before changing connectivity. +- [Identity and Certificates]() — Connect directory services, certificate trust, single sign-on, and application authentication. Start with the identity system involved, then follow the integration or maintenance procedure. +- [Automation]() — Connect source control, automation controllers, managed hosts, and configuration delivery. Use the documented execution environment and authentication method for each workflow. +- [Backup and Recovery]() — Find backup concepts, repository maintenance, and recovery dependencies. Select the procedure for the affected backup system and distinguish a backup restore from a replica or snapshot operation. +- [Applications]() — Find applications by the service they provide, then continue to their deployment, authentication, data, and maintenance documentation. +- [Windows and Linux]() — Find workstation and server operating-system setup, updates, and repairs. Storage, networking, and identity tasks are linked to their subject guides when they cross operating-system boundaries. +- [Foundations]() — Maintain the knowledgebase using the authoritative styling contract, document roles, and source-relative links. diff --git a/reference/infrastructure/hardware/index.md b/reference/infrastructure/hardware/index.md deleted file mode 100644 index 4cdc994..0000000 --- a/reference/infrastructure/hardware/index.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -tags: - - Infrastructure - - Hardware - - Index - - Documentation ---- - -# Hardware -## Purpose -Physical assets, node inventories, storage layouts, and power topology for the lab. - -## Includes -- Node build sheets and inventory -- Disk arrays and drive replacement procedures -- Power and UPS mapping - -## New Document Template -````markdown -# -## Purpose - - -!!! info "Assumptions" - - - - - -## Inventory -- - -## Procedure -```sh -# Commands (if applicable) -``` - -## Validation -- -```` diff --git a/reference/infrastructure/networking/index.md b/reference/infrastructure/networking/index.md deleted file mode 100644 index 7c779c0..0000000 --- a/reference/infrastructure/networking/index.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -tags: - - Infrastructure - - Networking - - Index - - Documentation ---- - -# Networking -## Purpose -Network topology, addressing, firewalling, VPN, and network service dependencies. - -## Includes -- IP tables and address plans -- Firewall and VPN configurations -- Network controllers and DNS-related services - -## New Document Template -````markdown -# -## Purpose - - -!!! info "Assumptions" - - - - - -## Architecture - - -## Procedure -```sh -# Commands or config steps -``` - -## Validation -- -```` diff --git a/scripts/Powershell/Exchange Online/Connect to Exchange Online.md b/scripts/Applications/Email/Exchange Online/Connect to Exchange Online.md similarity index 60% rename from scripts/Powershell/Exchange Online/Connect to Exchange Online.md rename to scripts/Applications/Email/Exchange Online/Connect to Exchange Online.md index a5cb580..43815a3 100644 --- a/scripts/Powershell/Exchange Online/Connect to Exchange Online.md +++ b/scripts/Applications/Email/Exchange Online/Connect to Exchange Online.md @@ -5,22 +5,28 @@ tags: - Scripting --- -**Purpose**: Sometimes you will need to connect to Office365 via powershell in order to perform troubleshooting / automation that either is too complex to do via the website, or is not exposed / possible to do via the website. +## Purpose +Sometimes you will need to connect to Office365 via powershell in order to perform troubleshooting / automation that either is too complex to do via the website, or is not exposed / possible to do via the website. ## Update Nuget Package Manager -``` powershell +```powershell Install-PackageProvider -Name NuGet -Force -ForceBootstrap ``` ## Install ExchangeOnlineManagement Powershell Modules You will need to install and import the modules for Exchange Online before you can run the commands necessary for interacting with it. -``` powershell + +```powershell Install-Module -Name ExchangeOnlineManagement -Force Import-Module ExchangeOnlineManagement ``` ## Connect to Exchange Online When you run the following command, it will open a dialog box to take the username, password, and MFA code (if applicable) for an administrative account in the Exchange Online environment. -``` powershell + +```powershell Connect-ExchangeOnline -UserPrincipalName admin@domain.com -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Email Documentation](<../../../../reference/Applications/Email/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Services/Email/Microsoft Exchange/Set Mailbox Auto Reply.md b/scripts/Applications/Email/Microsoft Exchange/Set Mailbox Auto Reply.md similarity index 66% rename from scripts/Services/Email/Microsoft Exchange/Set Mailbox Auto Reply.md rename to scripts/Applications/Email/Microsoft Exchange/Set Mailbox Auto Reply.md index 962b37b..59b58f7 100644 --- a/scripts/Services/Email/Microsoft Exchange/Set Mailbox Auto Reply.md +++ b/scripts/Applications/Email/Microsoft Exchange/Set Mailbox Auto Reply.md @@ -4,11 +4,12 @@ tags: - Email --- -**Purpose**: Sometimes you need to set an autoreply on a mailbox on behalf of someone else. In these cases, you can leverage the "Exchange Admin Shell" to configure an auto-reply to anyone who sends an email to the mailbox. +## Purpose +Sometimes you need to set an autoreply on a mailbox on behalf of someone else. In these cases, you can leverage the "Exchange Admin Shell" to configure an auto-reply to anyone who sends an email to the mailbox. In the example below, replace `` with the shortened username of the target user. (e.g. `nicole.rappe` not `nicole.rappe@bunny-lab.io`) -``` powershell +```powershell Set-MailboxAutoReplyConfiguration -Identity -AutoReplyState Scheduled -StartTime "1/1/2025 00:00:00" -EndTime "1/15/2025 00:00:00" -InternalMessage "Example,

Message here.

Thank you." -ExternalMessage "Example,

Message here.

Thank you." ``` @@ -17,10 +18,14 @@ Set-MailboxAutoReplyConfiguration -Identity -AutoReplyState Scheduled !!! example "Example Email Reply" The email auto reply will look something like this based on the command above. - ``` + + ```text Example, Message Here. Thank you. - ``` \ No newline at end of file + ``` + +## Related Documentation +- [Related Email Documentation](<../../../../reference/Applications/Email/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Applications/Email/Microsoft Exchange/Start Exchange Services.md b/scripts/Applications/Email/Microsoft Exchange/Start Exchange Services.md new file mode 100644 index 0000000..d537122 --- /dev/null +++ b/scripts/Applications/Email/Microsoft Exchange/Start Exchange Services.md @@ -0,0 +1,27 @@ +--- +tags: + - Microsoft Exchange + - Email +--- + +## Purpose +Start the Microsoft Exchange and supporting Windows services selected by this PowerShell script. The script sets their startup type to Automatic, so review the service requirements for the target server before running it. + +!!! warning "Review Service Startup Changes" + The preserved script sets every selected service to Automatic and starts it, including services that may intentionally be disabled. It does not perform a service restart. Do not use it as the return-to-service step for Exchange SE; follow the baseline-specific service validation in the rolling-update workflow. + +```powershell +$servicelist = Get-Service | Where-Object {$_.DisplayName -like "Microsoft Exchange *"} +$servicelist += Get-Service | Where-Object {$_.DisplayName -eq "IIS Admin Service"} +$servicelist += Get-Service | Where-Object { $_.DisplayName –eq "Windows Management Instrumentation" } +$servicelist += Get-Service | Where-Object { $_.DisplayName –eq "World Wide Web Publishing Service" } + +foreach($service in $servicelist){ + Set-Service $service -StartupType Automatic + Start-Service $service +} +``` + +## Related Documentation +- [Exchange SE Maintenance](<../../../../workflows/Applications/Email/Microsoft Exchange/Perform Exchange SE DAG Rolling Updates.md>) — Use its service-baseline validation when maintaining an SE DAG. +- [Related Email Documentation](<../../../../reference/Applications/Email/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Applications/Files and Collaboration/DFS/Report DFS Namespaces and Replication.md b/scripts/Applications/Files and Collaboration/DFS/Report DFS Namespaces and Replication.md new file mode 100644 index 0000000..36e320c --- /dev/null +++ b/scripts/Applications/Files and Collaboration/DFS/Report DFS Namespaces and Replication.md @@ -0,0 +1,281 @@ +--- +tags: + - DFS + - Active Directory + - PowerShell +--- + +## Purpose +Report the DFS namespaces, folder targets, and replication configuration in the current Active Directory domain. Run the script in PowerShell with the DFSN and DFSR modules and permission to query the domain. + +## Script +You may want to put together a simple table report of the DFS namespaces, replication info, and target folders. You can run the following powershell script to generate a nice table-based report of the current structure of the DFS namespaces in your domain. + +??? example "Powershell Reporting Script" + + ```powershell + # Automatically detect current AD domain and use it as DFS prefix + try { + $Domain = ([System.DirectoryServices.ActiveDirectory.Domain]::GetCurrentDomain()).Name + $DomainPrefix = "\\$Domain" + } catch { + Write-Warning "Unable to detect domain automatically. Falling back to manual value." + $DomainPrefix = "\\bunny-lab.io" + } + + Import-Module DFSN -ErrorAction Stop + Import-Module DFSR -ErrorAction Stop + + function Get-ServerNameFromPath { + param([string]$Path) + if ([string]::IsNullOrWhiteSpace($Path)) { return $null } + if ($Path -like "\\*") { return ($Path -split '\\')[2] } + return $null + } + function Get-Max3 { + param([int[]]$Values) + if (-not $Values) { return 0 } + return (($Values | Measure-Object -Maximum).Maximum) + } + + # Build: GroupName (lower) -> memberships[] + $allGroups = Get-DfsReplicationGroup -ErrorAction SilentlyContinue + $groupMembershipMap = @{} + foreach ($g in $allGroups) { + $ms = Get-DfsrMembership -GroupName $g.GroupName -ErrorAction SilentlyContinue + $groupMembershipMap[$g.GroupName.ToLower()] = $ms + } + + # Flatten all memberships for regex fallback + $allMemberships = @() + foreach ($arr in $groupMembershipMap.Values) { if ($arr) { $allMemberships += $arr } } + + $rows = New-Object System.Collections.Generic.List[psobject] + + # Enumerate namespace roots + $roots = Get-DfsnRoot -ErrorAction Stop | Where-Object { $_.Path -like "$DomainPrefix\*" } + + Write-Host "DFS Namespace and Replication Overview" -ForegroundColor Cyan + Write-Host "------------------------------------------------------`n" + + foreach ($root in $roots) { + + $rootPath = $root.Path + $rootLeaf = ($rootPath -split '\\')[-1] + + $nsServers = @() + $rootTargets = Get-DfsnRootTarget -Path $rootPath -ErrorAction SilentlyContinue + foreach ($rt in $rootTargets) { + $srv = Get-ServerNameFromPath $rt.TargetPath + if ($srv) { $nsServers += $srv } + } + + # Folders under this root + $folders = Get-DfsnFolder -Path "$rootPath\*" -ErrorAction SilentlyContinue | Sort-Object Path + + foreach ($f in $folders) { + $namespaceFull = $f.Path + $leaf = ($f.Path -split '\\')[-1] + + # DFSN folder targets + $targets = Get-DfsnFolderTarget -Path $f.Path -ErrorAction SilentlyContinue + $targets = @($targets | Sort-Object { Get-ServerNameFromPath $_.TargetPath }) # ensure array + + # Map to DFSR group by naming; fallback to regex on ContentPath + $candidateGroup = ((($rootPath -replace '^\\\\','') + '\' + $leaf).ToLower()) + if ($groupMembershipMap.ContainsKey($candidateGroup)) { + $msForFolder = $groupMembershipMap[$candidateGroup] + } else { + $escapedRootLeaf = [regex]::Escape($rootLeaf) + $escapedLeaf = [regex]::Escape($leaf) + $regex = "\\$escapedRootLeaf\\$escapedLeaf($|\\)" + $msForFolder = $allMemberships | Where-Object { $_.ContentPath -imatch $regex } + } + $msForFolder = @($msForFolder) # normalize to array + + # Build aligned rows: one per target + $targetLines = @() + $replLines = @() + + foreach ($t in $targets) { + $tServer = Get-ServerNameFromPath $t.TargetPath + $targetLines += $t.TargetPath + + $msForServer = $null + if ($msForFolder.Count -gt 0) { + $msForServer = $msForFolder | Where-Object { $_.ComputerName -ieq $tServer } | Select-Object -First 1 + } + if ($msForServer -and $msForServer.ContentPath) { $replLines += $msForServer.ContentPath } else { $replLines += '' } + } + + # Max line count for row expansion (PS 5.1 safe) + $maxLines = Get-Max3 @($targetLines.Count, $replLines.Count, $nsServers.Count) + + for ($i = 0; $i -lt $maxLines; $i++) { + + # Precompute values (PS 5.1: no inline-if in hashtables) + $nsVal = '' + if ($i -eq 0) { $nsVal = $namespaceFull } + + $targetVal = '' + if ($i -lt $targetLines.Count) { $targetVal = $targetLines[$i] } + + $replVal = '' + if ($i -lt $replLines.Count) { $replVal = $replLines[$i] } + + $nsServerVal = '' + if ($i -lt $nsServers.Count) { $nsServerVal = $nsServers[$i] } + + $row = [PSCustomObject]@{ + 'Namespace' = $nsVal + 'Member Folder Target(s)' = $targetVal + 'Replication Locations' = $replVal + 'Namespace Servers' = $nsServerVal + } + $rows.Add($row) | Out-Null + } + } + } + + # Render as a PowerShell bordered grid with one-space left/right padding in every cell + function Write-DfsGrid { + [CmdletBinding()] + param( + [Parameter(Mandatory)] + [System.Collections.IEnumerable]$Data, + + [string[]]$Columns = @('Namespace','Member Folder Target(s)','Replication Locations','Namespace Servers'), + + # Reasonable max widths; tune to your console (these are content+padding widths) + [int[]]$MaxWidths = @(70, 70, 52, 30), + + [switch]$Ascii # use +-| instead of box-drawing if your console garbles Unicode + ) + + # Ensure arrays align + if ($MaxWidths.Count -lt $Columns.Count) { + $pad = New-Object System.Collections.Generic.List[int] + $pad.AddRange($MaxWidths) + for ($i=$MaxWidths.Count; $i -lt $Columns.Count; $i++) { $pad.Add(40) } + $MaxWidths = $pad.ToArray() + } + + # Characters + if ($Ascii) { + $H = @{ tl='+'; tr='+'; bl='+'; br='+'; hz='-'; vt='|'; tj='+'; mj='+'; bj='+' } + } else { + # Box-drawing + $H = @{ tl='┌'; tr='┐'; bl='└'; br='┘'; hz='─'; vt='│'; tj='┬'; mj='┼'; bj='┴' } + try { [Console]::OutputEncoding = [Text.UTF8Encoding]::UTF8 } catch {} + } + + function TruncPad([string]$s, [int]$w) { + if ($null -eq $s) { $s = '' } + $s = $s -replace '\r','' -replace '\t',' ' + if ($s.Length -le $w) { return $s.PadRight($w, ' ') } + if ($w -le 1) { return $s.Substring(0, $w) } + return ($s.Substring(0, $w-1) + '…') + } + + # Materialize and compute widths (include one-space left/right padding for header and data) + $rows = @($Data | ForEach-Object { + $o = @{} + foreach ($c in $Columns) { $o[$c] = [string]($_.$c) } + [pscustomobject]$o + }) + + $widths = @() + for ($i=0; $i -lt $Columns.Count; $i++) { + $col = $Columns[$i] + # Start with header length including padding + $max = (" " + $col + " ").Length + foreach ($r in $rows) { + $len = (" " + [string]$r.$col + " ").Length + if ($len -gt $max) { $max = $len } + } + $widths += [Math]::Min($max, $MaxWidths[$i]) + } + + # Line builders + function DrawTop() { + $line = $H.tl + for ($i = 0; $i -lt $widths.Count; $i++) { + $line += ($H.hz * $widths[$i]) + if ($i -lt ($widths.Count - 1)) { + $line += $H.tj + } else { + $line += $H.tr + } + } + $line + } + function DrawMid([string[]]$Columns, [int[]]$widths, $H) { + $line = $H.vt + for ($i=0; $i -lt $widths.Count; $i++) { + $line += TruncPad (" " + $Columns[$i] + " ") $widths[$i] + $line += $H.vt + } + $line + } + function DrawSep() { + $line = $H.vt + for ($i=0; $i -lt $widths.Count; $i++) { + $line += ($H.hz * $widths[$i]) + $line += $H.vt + } + $line + } + function DrawHeaderSep() { + $line = $H.vt + for ($i=0; $i -lt $widths.Count; $i++) { + $line += ($H.hz * $widths[$i]) + $line += $H.vt + } + $line + } + function DrawBottom() { + $line = $H.bl + for ($i = 0; $i -lt $widths.Count; $i++) { + $line += ($H.hz * $widths[$i]) + if ($i -lt ($widths.Count - 1)) { + $line += $H.bj + } else { + $line += $H.br + } + } + $line + } + function DrawRow($r, [string[]]$Columns, [int[]]$widths, $H) { + $line = $H.vt + for ($i=0; $i -lt $widths.Count; $i++) { + $val = [string]$r.($Columns[$i]) + $line += TruncPad (" " + $val + " ") $widths[$i] + $line += $H.vt + } + $line + } + + # Render with group separators between namespaces (when the Namespace cell is non-empty) + Write-Host (DrawTop) + Write-Host (DrawMid -Columns $Columns -widths $widths -H $H) + Write-Host (DrawHeaderSep) + + $first = $true + foreach ($r in $rows) { + if (-not $first -and ([string]$r.$($Columns[0])) ) { + # Namespace changed → draw a separator + Write-Host (DrawSep) + } + $first = $false + Write-Host (DrawRow -r $r -Columns $Columns -widths $widths -H $H) + } + + Write-Host (DrawBottom) + } + + Write-DfsGrid -Data $rows + ``` + +## Related Documentation +- [DFS Deployment](<../../../../deployments/Applications/Files and Collaboration/Windows Server/DFS Namespaces with Replication.md>) — Review the namespace and replication structure that this report inspects. +- [Related Files and Collaboration Documentation](<../../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Applications/Files and Collaboration/DFS/Report DFS Replication Backlog.md b/scripts/Applications/Files and Collaboration/DFS/Report DFS Replication Backlog.md new file mode 100644 index 0000000..d3f782f --- /dev/null +++ b/scripts/Applications/Files and Collaboration/DFS/Report DFS Replication Backlog.md @@ -0,0 +1,132 @@ +--- +tags: + - DFS + - Replication + - PowerShell +--- + +## Purpose +Report the directional replication backlog between the DFS member servers in the script. Set the member names to the intended deployment and run from a PowerShell session with access to its DFS replication configuration. + +## Script +You may want to check that replication is occurring bi-directionally between every member server in your DFS deployment. I wrote a script below that effectively shows you every replication group and each directional backlog status. + +```powershell +# --- CONFIG --- +$Members = @("LAB-FPS-01","LAB-FPS-02") +$SummarizeAcrossFolders = $true # $true = one line per direction per RG; $false = per-folder lines + +function Invoke-DfsrBacklogStatus { + param( + [Parameter(Mandatory)] [string] $RG, + [Parameter(Mandatory)] [string] $RF, + [Parameter(Mandatory)] [string] $Send, + [Parameter(Mandatory)] [string] $Recv + ) + + $out = & dfsrdiag backlog /rgname:"$RG" /rfname:"$RF" /sendingmember:"$Send" /receivingmember:"$Recv" 2>&1 | Out-String + $outTrim = ($out -split "`r?`n" | ForEach-Object { $_.Trim() }) | Where-Object { $_ -ne "" } + + if ($out -match 'No Backlog') { + return [pscustomobject]@{ Status="No Backlog"; Count=0; Detail=$null } + } + + $count = $null + $countLine = $outTrim | Where-Object { $_ -match '(?i)backlog' } | Select-Object -First 1 + if ($countLine -and ($countLine -match '(\d+)')) { $count = [int]$matches[1] } + + $detail = ($outTrim | Select-Object -First 8) -join " | " + + return [pscustomobject]@{ + Status = if ($count -ne $null) { "Backlog: $count" } else { "Backlog/Check Output" } + Count = $count + Detail = $detail + } +} + +$groups = Get-DfsReplicationGroup | Sort-Object GroupName + +foreach ($g in $groups) { + $rg = $g.GroupName + $rfs = Get-DfsReplicatedFolder -GroupName $rg | Sort-Object FolderName + + Write-Host "" + Write-Host ("== Replication Group: {0} ==" -f $rg) + + foreach ($send in $Members) { + foreach ($recv in $Members) { + if ($send -eq $recv) { continue } + + if ($SummarizeAcrossFolders) { + $worstCount = 0 + $nonZero = @() + $errorsOrDetails = @() + + foreach ($rfObj in $rfs) { + $rf = $rfObj.FolderName + $res = Invoke-DfsrBacklogStatus -RG $rg -RF $rf -Send $send -Recv $recv + + if ($res.Status -ne "No Backlog") { + $nonZero += [pscustomobject]@{ RF=$rf; Status=$res.Status; Count=$res.Count; Detail=$res.Detail } + if ($res.Count -ne $null -and $res.Count -gt $worstCount) { $worstCount = $res.Count } + + # ✅ FIX: ${rf} avoids the ':' parsing issue + if ($res.Detail) { $errorsOrDetails += "RF=${rf}: $($res.Detail)" } + } + } + + if ($nonZero.Count -eq 0) { + Write-Host ("{0} -> {1}: No Backlog" -f $send, $recv) + } else { + if ($worstCount -gt 0) { + Write-Host ("{0} -> {1}: Backlog (max {2} across RFs)" -f $send, $recv, $worstCount) + } else { + Write-Host ("{0} -> {1}: Backlog/Errors (see details)" -f $send, $recv) + } + + $errorsOrDetails | Select-Object -First 5 | ForEach-Object { Write-Host (" - {0}" -f $_) } + if ($errorsOrDetails.Count -gt 5) { Write-Host " - ... (more omitted)" } + } + } + else { + foreach ($rfObj in $rfs) { + $rf = $rfObj.FolderName + $res = Invoke-DfsrBacklogStatus -RG $rg -RF $rf -Send $send -Recv $recv + + if ($res.Status -eq "No Backlog") { + Write-Host ("{0} -> {1} [{2}]: No Backlog" -f $send, $recv, $rf) + } else { + Write-Host ("{0} -> {1} [{2}]: {3}" -f $send, $recv, $rf, $res.Status) + if ($res.Detail) { Write-Host (" - {0}" -f $res.Detail) } + } + } + } + } + } +} +``` + +!!! example "Example Output" + You will see output like the following when you run the script. + + ```powershell + == Replication Group: bunny-lab.io\music\fl studio plugins == + LAB-FPS-01 -> LAB-FPS-02: No Backlog + LAB-FPS-02 -> LAB-FPS-01: No Backlog + + == Replication Group: bunny-lab.io\music\personal music == + LAB-FPS-01 -> LAB-FPS-02: No Backlog + LAB-FPS-02 -> LAB-FPS-01: No Backlog + + == Replication Group: bunny-lab.io\music\shared music == + LAB-FPS-01 -> LAB-FPS-02: No Backlog + LAB-FPS-02 -> LAB-FPS-01: No Backlog + + == Replication Group: bunny-lab.io\projects\coding == + LAB-FPS-01 -> LAB-FPS-02: No Backlog + LAB-FPS-02 -> LAB-FPS-01: No Backlog + ``` + +## Related Documentation +- [DFS Deployment](<../../../../deployments/Applications/Files and Collaboration/Windows Server/DFS Namespaces with Replication.md>) — Identify the replication members and folders. +- [Related Files and Collaboration Documentation](<../../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/General Purpose/Directory Walker.md b/scripts/Applications/Files and Collaboration/Directory Walker.md similarity index 82% rename from scripts/Powershell/General Purpose/Directory Walker.md rename to scripts/Applications/Files and Collaboration/Directory Walker.md index 0a12e4c..0c7efe2 100644 --- a/scripts/Powershell/General Purpose/Directory Walker.md +++ b/scripts/Applications/Files and Collaboration/Directory Walker.md @@ -4,7 +4,7 @@ tags: - Scripting --- -**Purpose**: +## Purpose Sometimes you just need a basic script that outputs a pretty directory and file tree. This script offers files and folders to ignore, and outputs a fancy directory tree. ```powershell @@ -26,8 +26,8 @@ function Export-Tree { [string]$Prefix ) - $items = Get-ChildItem -Path $Folder -Force | Where-Object { - $_.Name -ne "." -and $_.Name -ne ".." -and + $items = Get-ChildItem -Path $Folder -Force | Where-Object { + $_.Name -ne "." -and $_.Name -ne ".." -and ($global:IgnoreList -notcontains $_.Name) } | Sort-Object PSIsContainer, Name @@ -51,4 +51,7 @@ function Export-Tree { # Run it Export-Tree -Path "." -OutFile "directory_tree.txt" -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Files and Collaboration Documentation](<../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/General Purpose/File Finder.md b/scripts/Applications/Files and Collaboration/File Finder.md similarity index 85% rename from scripts/Powershell/General Purpose/File Finder.md rename to scripts/Applications/Files and Collaboration/File Finder.md index 77d5b1f..0cefa97 100644 --- a/scripts/Powershell/General Purpose/File Finder.md +++ b/scripts/Applications/Files and Collaboration/File Finder.md @@ -5,10 +5,10 @@ tags: - Scripting --- -**Purpose**: +## Purpose Locate specific files, and copy them with a renamed datestamp appended to a specific directory. -``` powershell +```powershell # Define an array of objects, each having a prefix and a suffix $files = @( @{Prefix="name"; Suffix="Extension"}, @@ -41,4 +41,7 @@ foreach ($file in $files) { Copy-Item -Path $match.FullName -Destination (Join-Path -Path $destination -ChildPath $newName) } } -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Files and Collaboration Documentation](<../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/SMB/Detailed Permission Report All Shares.md b/scripts/Applications/Files and Collaboration/SMB/Report NTFS Permissions Across Shares.md similarity index 77% rename from scripts/Powershell/SMB/Detailed Permission Report All Shares.md rename to scripts/Applications/Files and Collaboration/SMB/Report NTFS Permissions Across Shares.md index 4f0523e..d5154cb 100644 --- a/scripts/Powershell/SMB/Detailed Permission Report All Shares.md +++ b/scripts/Applications/Files and Collaboration/SMB/Report NTFS Permissions Across Shares.md @@ -6,10 +6,10 @@ tags: - Scripting --- -**Purpose**: +## Purpose This script will iterate over all network shares hosted by the computer it is running on, and will give *recursive* permissions to all folders, subfolders, and files, including hidden ones. It is very I/O intensive given it iterates recursively on every file/folder being shared. -``` powershell +```powershell $AllShares = Get-SMBShare | Where-Object {$_.Description -NotMatch "Default share|Remote Admin|Remote IPC|Printer Drivers"} | Select-Object -ExpandProperty Path $Output = @() ForEach ($SMBDirectory in $AllShares) @@ -17,13 +17,16 @@ ForEach ($SMBDirectory in $AllShares) $FolderPath = Get-ChildItem -Directory -Path $SMBDirectory -Recurse -Force ForEach ($Folder in $FolderPath) { $Acl = Get-Acl -Path $Folder.FullName - ForEach ($Access in $Acl.Access) + ForEach ($Access in $Acl.Access) { $Properties = [ordered]@{'Folder Name'=$Folder.FullName;'Group/User'=$Access.IdentityReference;'Permissions'=$Access.FileSystemRights;'Inherited'=$Access.IsInherited} - $Output += New-Object -TypeName PSObject -Property $Properties + $Output += New-Object -TypeName PSObject -Property $Properties } } } $Output | Export-CSV -Path C:\SMB_REPORT.csv -NoTypeInformation -Append -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Files and Collaboration Documentation](<../../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/SMB/Top Level Permission Report All Shares.md b/scripts/Applications/Files and Collaboration/SMB/Report SMB Share Permissions.md similarity index 70% rename from scripts/Powershell/SMB/Top Level Permission Report All Shares.md rename to scripts/Applications/Files and Collaboration/SMB/Report SMB Share Permissions.md index ced1dbd..66e9ed8 100644 --- a/scripts/Powershell/SMB/Top Level Permission Report All Shares.md +++ b/scripts/Applications/Files and Collaboration/SMB/Report SMB Share Permissions.md @@ -6,13 +6,16 @@ tags: - Scripting --- -**Purpose**: +## Purpose This script will iterate over all network shares hosted by the computer it is running on, and will give *top-level* permissions to all the shared folders. It will not navigate deeper than the top-level in its report. Very I/O friendly. -``` powershell +```powershell $AllShares = Get-SMBShare | Where-Object {$_.Description -NotMatch "Default share|Remote Admin|Remote IPC|Printer Drivers"} | Select-Object -ExpandProperty Name ForEach ($SMBDirectory in $AllShares) { Get-SMBShareAccess -Name $SMBDirectory | Export-CSV -Path C:\SMB_REPORT.csv -NoTypeInformation -Append } -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Files and Collaboration Documentation](<../../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/Nextcloud/Upload Data to Nextcloud Share.md b/scripts/Applications/Files and Collaboration/Upload Data to a Nextcloud Share.md similarity index 92% rename from scripts/Powershell/Nextcloud/Upload Data to Nextcloud Share.md rename to scripts/Applications/Files and Collaboration/Upload Data to a Nextcloud Share.md index 4f2b15b..705e155 100644 --- a/scripts/Powershell/Nextcloud/Upload Data to Nextcloud Share.md +++ b/scripts/Applications/Files and Collaboration/Upload Data to a Nextcloud Share.md @@ -5,12 +5,13 @@ tags: - Scripting --- -**Purpose**: In some unique cases, you want to be able to either perform backups of data or exfiltrate data to Nextcloud from a local device via the use of a script. Doing such a thing with Nextcloud as the destination is not very documented, but you can achieve that result by running a script like what is seen below: +## Purpose +In some unique cases, you want to be able to either perform backups of data or exfiltrate data to Nextcloud from a local device via the use of a script. Doing such a thing with Nextcloud as the destination is not very documented, but you can achieve that result by running a script like what is seen below: ## Windows !!! abstract "Environment Variables" - You will need to assign the following variables either within the script or externally via environment variables at the time the script is executed. - + You will need to assign the following variables either within the script or externally via environment variables at the time the script is executed. + | **Variable** | **Default Value** | **Description** | | :--- | :--- | :--- | | `NEXTCLOUD_SERVER_URL` | `https://cloud.bunny-lab.io` | This is the base URL of the Nextcloud server that data will be copied to. | @@ -22,7 +23,7 @@ tags: | `LOGFILE` | `C:\Windows\Temp\nc_pull.log` | This file is how the script has "persistence". In case the computer is shut down, rebooted, etc, when it comes back online and the script is re-ran against it, it reads this file to pick up where it last was, and attempts to resume from that point. If this transfer is meant to be hidden, put this file somewhere someone is not likely to find it easily. | ### Powershell Script -``` powershell +```powershell # -------------------------- # Function for File Upload Logic # -------------------------- @@ -59,7 +60,7 @@ Function Upload-Files ($targetDir) { # Record this file in the log since it was successfully uploaded Add-Content -Path $LOGFILE -Value $_.FullName - + } else { Write-Host "Skipping previously uploaded file $($_.FullName)" } @@ -99,8 +100,8 @@ Function Upload-Files ($targetDir) { ## MacOS/Linux !!! abstract "Environment Variables" - You will need to assign the following variables either within the script or externally via environment variables at the time the script is executed. - + You will need to assign the following variables either within the script or externally via environment variables at the time the script is executed. + | **Variable** | **Default Value** | **Description** | | :--- | :--- | :--- | | `NEXTCLOUD_SERVER_URL` | `https://cloud.bunny-lab.io` | This is the base URL of the Nextcloud server that data will be copied to. | @@ -109,8 +110,8 @@ Function Upload-Files ($targetDir) { | `DATA_TO_COPY` | `/home/bunny/example` | This directory target is the primary focus of the upload / backup / exfiltration. The script will iterate through this target first before it moves onto the secondary target. The target can be a directory or a single file. This will act as the main priority of the transfer. | | `LOGFILE` | `/tmp/uploaded_files.log` | This file is how the script has "persistence". In case the computer is shut down, rebooted, etc, when it comes back online and the script is re-ran against it, it reads this file to pick up where it last was, and attempts to resume from that point. If this transfer is meant to be hidden, put this file somewhere someone is not likely to find it easily. | -### Bash Script -``` sh +### Bash Script +```sh #!/bin/bash # Directory to search @@ -128,7 +129,7 @@ fi find "$DIR" -type f -print0 | while IFS= read -r -d '' file; do # Extract just the filename filename=$(basename "$file") - + # Check if this file has been uploaded before if ! grep -q "$file" "$LOGFILE"; then echo "Uploading $file ..." @@ -153,4 +154,7 @@ find "$DIR" -type f -print0 | while IFS= read -r -d '' file; do echo "Skipping previously uploaded file $file" fi done -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Files and Collaboration Documentation](<../../../reference/Applications/Files and Collaboration/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/Minecraft Server/Update Script.md b/scripts/Applications/Gaming and Media/Update the ATM10 Minecraft Server.md similarity index 94% rename from scripts/Powershell/Minecraft Server/Update Script.md rename to scripts/Applications/Gaming and Media/Update the ATM10 Minecraft Server.md index 99f9569..4065377 100644 --- a/scripts/Powershell/Minecraft Server/Update Script.md +++ b/scripts/Applications/Gaming and Media/Update the ATM10 Minecraft Server.md @@ -5,7 +5,8 @@ tags: - Scripting --- -**Purpose**: This script was purpose-built for the homelab Minecraft servers in my homelab. It may need to be ported based on your own needs. +## Purpose +This script was purpose-built for the homelab Minecraft servers in my homelab. It may need to be ported based on your own needs. ```powershell clear @@ -34,7 +35,7 @@ function Get-ZipFileName { Write-Host "File not found! Please check the file name and try again." -ForegroundColor Red exit } - + Write-Host "ZIP file found: $zipFilePath" -ForegroundColor Green return $zipFilePath } @@ -103,7 +104,7 @@ function Rename-OldServer { $currentDate = Get-Date -Format "MM-dd-yyyy" $backupFolderPath = "$oldFolderPath.backup.$currentDate" - + Write-Host "Step 4: Renaming old server folder to: $backupFolderPath" -ForegroundColor Yellow Rename-Item -Path $oldFolderPath -NewName $backupFolderPath Write-Host "Old server folder renamed to: $backupFolderPath" -ForegroundColor Green @@ -158,4 +159,7 @@ Write-Host "Press any key to exit the script" [System.Console]::ReadKey($true) # Waits for a key press and doesn't display the pressed key clear -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/batch/Blue Iris/Server Watchdog.md b/scripts/Applications/Home Automation/Blue Iris Server Watchdog.md similarity index 61% rename from scripts/batch/Blue Iris/Server Watchdog.md rename to scripts/Applications/Home Automation/Blue Iris Server Watchdog.md index bbee481..b5e391e 100644 --- a/scripts/batch/Blue Iris/Server Watchdog.md +++ b/scripts/Applications/Home Automation/Blue Iris Server Watchdog.md @@ -7,7 +7,10 @@ tags: - Windows --- -``` batch +## Purpose +This document records the procedure for blue iris server watchdog. Follow the environment assumptions and commands below. + +```batch @echo off REM Change to the Blue Iris 5 directory @@ -28,4 +31,7 @@ timeout /t 10 /nobreak >nul REM Go back to the beginning of the loop GOTO :LOOP -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Applications Documentation](<../../../reference/Applications/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Applications/index.md b/scripts/Applications/index.md new file mode 100644 index 0000000..53f7f9e --- /dev/null +++ b/scripts/Applications/index.md @@ -0,0 +1,19 @@ +--- +tags: + - Applications + - Scripts + - Documentation +--- + +# Applications +## Purpose +Find scripts for applications. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- Email +- Files and Collaboration +- Gaming and Media +- Home Automation + +## Follow the Subject +[Applications](<../../reference/Applications/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/scripts/Bash/Git Repo Updater.md b/scripts/Automation/Gitea/Git Repo Updater.md similarity index 83% rename from scripts/Bash/Git Repo Updater.md rename to scripts/Automation/Gitea/Git Repo Updater.md index 22237c4..6ca8964 100644 --- a/scripts/Bash/Git Repo Updater.md +++ b/scripts/Automation/Gitea/Git Repo Updater.md @@ -5,11 +5,10 @@ tags: - Linux --- -# Git Repo Updater (Script) ## Purpose -Standalone `repo_watcher.sh` script used by the Git Repo Updater container. This script clones or pulls one or more repositories and rsyncs them into destination paths. +Standalone `repo_watcher.sh` script used by the Git Repo Updater container. This script clones or pulls one or more repositories and rsyncs them into destination paths. -For the containerized version and deployment details, see the [Git Repo Updater container doc](../../deployments/platforms/containerization/docker/custom-containers/git-repo-updater.md). +For the containerized version and deployment details, see the [Git Repo Updater container doc](<../../../deployments/Containers/Docker/Git Repo Updater.md>). ## Script ```sh @@ -67,3 +66,5 @@ while true; do done ``` +## Related Documentation +- [Related Gitea Workflows](<../../../reference/Automation/Gitea Configuration Delivery.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Automation/index.md b/scripts/Automation/index.md new file mode 100644 index 0000000..a96aa04 --- /dev/null +++ b/scripts/Automation/index.md @@ -0,0 +1,16 @@ +--- +tags: + - Automation + - Scripts + - Documentation +--- + +# Automation +## Purpose +Find scripts for automation. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- Gitea + +## Follow the Subject +[Automation](<../../reference/Automation/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/scripts/Bash/Fix Displaylink Issues on Linux.md b/scripts/Bash/Fix Displaylink Issues on Linux.md deleted file mode 100644 index 757eb82..0000000 --- a/scripts/Bash/Fix Displaylink Issues on Linux.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -tags: - - Linux - - Bash - - Scripting ---- - -``` sh -xrandr --auto -xrandr --setprovideroutputsource 4 0 -xrandr --output HDMI-1 --primary --mode 1920x1080 --rate 75.00 --output DVI-I-1-1 --mode 1920x1080 --rate 60.00 --right-of HDMI-1 --output -eDP-1 --off -``` \ No newline at end of file diff --git a/scripts/Bash/Mdadm Grow Array Size.md b/scripts/Bash/Mdadm Grow Array Size.md deleted file mode 100644 index 9e277c2..0000000 --- a/scripts/Bash/Mdadm Grow Array Size.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -tags: - - RAID - - Bash - - Scripting - - Linux ---- - -https://www.digitalocean.com/community/tutorials/how-to-create-raid-arrays-with-mdadm-on-ubuntu-16-04 -``` sh -sudo mdadm --grow /dev/md0 -l 5 -cat /proc/mdstat -``` \ No newline at end of file diff --git a/scripts/Bash/Open Port Checker.md b/scripts/Bash/Open Port Checker.md deleted file mode 100644 index 25b2012..0000000 --- a/scripts/Bash/Open Port Checker.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -tags: - - Bash - - Ports - - Scripting - - Linux ---- - -**Purpose**: -If you want to check if a certain TCP port is open on a server. - -## Netcat Command -``` sh -netcat -z -n -v -``` \ No newline at end of file diff --git a/scripts/Bash/Transfer Files with Netcat.md b/scripts/Bash/Transfer Files with Netcat.md deleted file mode 100644 index 18e1fa8..0000000 --- a/scripts/Bash/Transfer Files with Netcat.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -tags: - - Bash - - Netcat - - File Transfer - - Scripting - - Linux ---- - -**Purpose**: You may find that you need to transfer a file, such as a public SSH key, or some other kind of file between two devices. In this scenario, we assume both devices have the `netcat` command available to them. By putting a network listener on the device recieving the file, then sending the file to that device's IP and port, you can successfully transfer data between computers without needing to set up SSH, FTP, or anything else to establish initial trust between the devices. [Original Reference Material](https://www.youtube.com/shorts/1j17UBGqSog). - -!!! warning - The data being transferred will not be encrypted. If you are transferring relatively-safe files such as public SSH keys, etc, this should be fine. - -### Destination Computer -Run the following command on the computer that will be recieving the file. -``` sh -netcat -l > /tmp/OUTPUT-AS-FILE.txt -``` - -### Source Computer -Run the following command on the computer that will be sending the file to the destination computer. -``` sh -cat INPUT-DATA.txt | netcat -q 0 -``` - -!!! info - The `-q 0` command argument causes the netcat connection to close itself automatically when the transfer is complete. \ No newline at end of file diff --git a/scripts/Identity and Certificates/Active Directory/Force Group Policy Updates Across the Domain.md b/scripts/Identity and Certificates/Active Directory/Force Group Policy Updates Across the Domain.md new file mode 100644 index 0000000..403eaa4 --- /dev/null +++ b/scripts/Identity and Certificates/Active Directory/Force Group Policy Updates Across the Domain.md @@ -0,0 +1,17 @@ +--- +tags: + - Group Policy + - PowerShell + - Scripting +--- + +## Purpose +This document records the procedure for force group policy updates across the domain. Follow the environment assumptions and commands below. + +```powershell +$computers = Get-ADComputer -Filter * -SearchBase "OU=Computers,DC=bunny-lab,DC=io" +$computers | ForEach-Object -Process {Invoke-GPUpdate -Computer $_.name -RandomDelayInMinutes 0 -Force} +``` + +## Related Documentation +- [Related Identity and Certificates Documentation](<../../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/Reporting/Get Password Expiration.md b/scripts/Identity and Certificates/Active Directory/Reports/Get Password Expiration.md similarity index 59% rename from scripts/Powershell/Reporting/Get Password Expiration.md rename to scripts/Identity and Certificates/Active Directory/Reports/Get Password Expiration.md index 0ba2b43..60c3f28 100644 --- a/scripts/Powershell/Reporting/Get Password Expiration.md +++ b/scripts/Identity and Certificates/Active Directory/Reports/Get Password Expiration.md @@ -5,9 +5,12 @@ tags: - Scripting --- -**Purpose**: +## Purpose Sometimes you need a report of every user in a domain, and if/when their passwords will expire. This one-liner command will help automate that reporting. -``` powershell +```powershell Get-Aduser -filter "enabled -eq 'true'" -properties passwordlastset, passwordneverexpires | ft Name, passwordlastset, Passwordneverexpires > C:\PWReport.txt -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Identity and Certificates Documentation](<../../../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Identity and Certificates/Active Directory/Reports/Inactive Computers.md b/scripts/Identity and Certificates/Active Directory/Reports/Inactive Computers.md new file mode 100644 index 0000000..a45fa11 --- /dev/null +++ b/scripts/Identity and Certificates/Active Directory/Reports/Inactive Computers.md @@ -0,0 +1,18 @@ +--- +tags: + - PowerShell + - Reporting + - Scripting +--- + +## Purpose +This document records the procedure for inactive computers. Follow the environment assumptions and commands below. + +```powershell +$DaysInactive = 30 +$time = (Get-Date).Adddays(-($DaysInactive)) +Get-ADComputer -Filter {LastLogonTimeStamp -lt $time} -ResultPageSize 2000 -resultSetSize $null -Properties Name | Select Name +``` + +## Related Documentation +- [Related Identity and Certificates Documentation](<../../../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Identity and Certificates/Active Directory/Reports/Inactive Users.md b/scripts/Identity and Certificates/Active Directory/Reports/Inactive Users.md new file mode 100644 index 0000000..b27c6ac --- /dev/null +++ b/scripts/Identity and Certificates/Active Directory/Reports/Inactive Users.md @@ -0,0 +1,19 @@ +--- +tags: + - PowerShell + - Reporting + - Scripting +--- + +## Purpose +This document records the procedure for inactive users. Follow the environment assumptions and commands below. + +```powershell +InactiveDays = 30 +$Days = (Get-Date).Adddays(-($InactiveDays)) +Get-ADUser -Filter {LastLogonTimeStamp -lt $Days -and enabled -eq $true} -Properties LastLogonTimeStamp | +select-object Name,@{Name="Date"; Expression={[DateTime]::FromFileTime($_.lastLogonTimestamp).ToString('MM-dd-yyyy')}} +``` + +## Related Documentation +- [Related Identity and Certificates Documentation](<../../../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/Azure/Check Email Aliases.md b/scripts/Identity and Certificates/Microsoft 365/Check Email Aliases.md similarity index 50% rename from scripts/Powershell/Azure/Check Email Aliases.md rename to scripts/Identity and Certificates/Microsoft 365/Check Email Aliases.md index 02630ff..1fa1211 100644 --- a/scripts/Powershell/Azure/Check Email Aliases.md +++ b/scripts/Identity and Certificates/Microsoft 365/Check Email Aliases.md @@ -5,17 +5,24 @@ tags: - Scripting --- -!!! info "Prerequesite: [Connect to Azure AD](./connect-to-azure-ad.md)" +## Purpose +This document records the procedure for check email aliases. Follow the environment assumptions and commands below. + +!!! info "Prerequesite: [Connect to Azure AD]()" The uppercase `SMTP` address is the primary address, while lowercase `smtp` are aliases. You can find the value in active directory in **"User > Attribute Editor > proxyAddresses"**. -``` powershell + +```powershell Get-AzureADUser -ObjectId "user@domain.com" | Select -Property ProxyAddresses ``` !!! example "Example Output" - ``` powershell + + ```powershell smtp:alias@domain.com smtp:alias@domain.onmicrosoft.com SMTP:primaryaddress@domain.com ``` +## Related Documentation +- [Related Identity and Certificates Documentation](<../../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/Azure/Connect to Azure AD.md b/scripts/Identity and Certificates/Microsoft 365/Connect to Azure AD.md similarity index 58% rename from scripts/Powershell/Azure/Connect to Azure AD.md rename to scripts/Identity and Certificates/Microsoft 365/Connect to Azure AD.md index 9752b08..6508ae7 100644 --- a/scripts/Powershell/Azure/Connect to Azure AD.md +++ b/scripts/Identity and Certificates/Microsoft 365/Connect to Azure AD.md @@ -4,21 +4,27 @@ tags: - Scripting --- -**Purpose**: Sometimes you will need to connect to Azure AD via powershell in order to perform troubleshooting / automation. +## Purpose +Sometimes you will need to connect to Azure AD via powershell in order to perform troubleshooting / automation. ## Update Nuget Package Manager -``` powershell +```powershell Install-PackageProvider -Name NuGet -Force -ForceBootstrap ``` ## Install AzureAD Powershell Modules You will need to install the modules for AzureAD before you can run the commands necessary for querying Azure. -``` powershell + +```powershell Install-Module -Name AzureAD ``` ## Connect to AzureAD When you run the following command, it will open a dialog box to take the username, password, and MFA code (if applicable) for an administrative account in the Azure Active Directory. -``` powershell + +```powershell Connect-AzureAD -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Identity and Certificates Documentation](<../../../reference/Identity and Certificates/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Identity and Certificates/index.md b/scripts/Identity and Certificates/index.md new file mode 100644 index 0000000..1f517ea --- /dev/null +++ b/scripts/Identity and Certificates/index.md @@ -0,0 +1,17 @@ +--- +tags: + - Identity and Certificates + - Scripts + - Documentation +--- + +# Identity and Certificates +## Purpose +Find scripts for identity and certificates. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- Active Directory +- Microsoft 365 + +## Follow the Subject +[Identity and Certificates](<../../reference/Identity and Certificates/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/scripts/Powershell/General Purpose/Remotely Change DNS Records.md b/scripts/Networking and Access/Change DNS Client Server Settings Remotely.md similarity index 62% rename from scripts/Powershell/General Purpose/Remotely Change DNS Records.md rename to scripts/Networking and Access/Change DNS Client Server Settings Remotely.md index f195819..b3c187b 100644 --- a/scripts/Powershell/General Purpose/Remotely Change DNS Records.md +++ b/scripts/Networking and Access/Change DNS Client Server Settings Remotely.md @@ -1,5 +1,11 @@ +--- +tags: + - Networking and Access + - Change DNS Client Server Settings Remotely +--- + ## Purpose -You may find that for one reason or another, you need to change DNS records of remote windows devices and cannot login via RDP or via console, yet somehow WinRM continues to work. In these scenarios, you can use the following commands to identify the network adapter, change its DNS servers, and verify the settings afterwards. +Change the DNS server addresses used by the selected Windows network adapters through PowerShell remoting. This changes client resolver settings rather than records hosted in a DNS zone. !!! info "Run as Domain Admin" You need to run the following commands within the context of a powershell session running as a domain admin, otherwise the `Invoke-Command` commands will fail to execute. @@ -16,4 +22,7 @@ Invoke-Command -ComputerName $DEVICE -ScriptBlock { Get-DnsClientServerAddress - # Replace Current DNS Servers for the selected interface Invoke-Command -ComputerName $DEVICE -ScriptBlock { Set-DnsClientServerAddress -InterfaceAlias "Ethernet" -ServerAddresses ("192.168.3.25","192.168.3.26") } -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Networking and Access Documentation](<../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Networking and Access/Check Open Ports.md b/scripts/Networking and Access/Check Open Ports.md new file mode 100644 index 0000000..b8b5d8a --- /dev/null +++ b/scripts/Networking and Access/Check Open Ports.md @@ -0,0 +1,18 @@ +--- +tags: + - Bash + - Ports + - Scripting + - Linux +--- + +## Purpose +If you want to check if a certain TCP port is open on a server. + +## Netcat Command +```sh +netcat -z -n -v +``` + +## Related Documentation +- [Related Networking and Access Documentation](<../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/General Purpose/DNS Hierarchy Correction.md b/scripts/Networking and Access/Correct DNS Server Priority.md similarity index 95% rename from scripts/Powershell/General Purpose/DNS Hierarchy Correction.md rename to scripts/Networking and Access/Correct DNS Server Priority.md index dbb72cd..1596c02 100644 --- a/scripts/Powershell/General Purpose/DNS Hierarchy Correction.md +++ b/scripts/Networking and Access/Correct DNS Server Priority.md @@ -98,4 +98,7 @@ foreach ($adapter in $adapters) { } Write-Host "DNS check and correction completed for adapters with a default gateway." -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Networking and Access Documentation](<../../reference/Networking and Access/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Networking and Access/index.md b/scripts/Networking and Access/index.md new file mode 100644 index 0000000..4e1f51f --- /dev/null +++ b/scripts/Networking and Access/index.md @@ -0,0 +1,18 @@ +--- +tags: + - Networking and Access + - Scripts + - Documentation +--- + +# Networking and Access +## Purpose +Find scripts for networking and access. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- Change DNS Client Server Settings Remotely +- Check Open Ports +- Correct DNS Server Priority + +## Follow the Subject +[Networking and Access](<../../reference/Networking and Access/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/scripts/Powershell/General Purpose/Force GPUpdate Domain Wide.md b/scripts/Powershell/General Purpose/Force GPUpdate Domain Wide.md deleted file mode 100644 index be3591d..0000000 --- a/scripts/Powershell/General Purpose/Force GPUpdate Domain Wide.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -tags: - - Group Policy - - PowerShell - - Scripting ---- - -``` powershell -$computers = Get-ADComputer -Filter * -SearchBase "OU=Computers,DC=bunny-lab,DC=io" -$computers | ForEach-Object -Process {Invoke-GPUpdate -Computer $_.name -RandomDelayInMinutes 0 -Force} -``` \ No newline at end of file diff --git a/scripts/Powershell/Reporting/Inactive Computers.md b/scripts/Powershell/Reporting/Inactive Computers.md deleted file mode 100644 index 69bf816..0000000 --- a/scripts/Powershell/Reporting/Inactive Computers.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -tags: - - PowerShell - - Reporting - - Scripting ---- - -``` powershell -$DaysInactive = 30 -$time = (Get-Date).Adddays(-($DaysInactive)) -Get-ADComputer -Filter {LastLogonTimeStamp -lt $time} -ResultPageSize 2000 -resultSetSize $null -Properties Name | Select Name -``` \ No newline at end of file diff --git a/scripts/Powershell/Reporting/Inactive Users.md b/scripts/Powershell/Reporting/Inactive Users.md deleted file mode 100644 index 37ae35b..0000000 --- a/scripts/Powershell/Reporting/Inactive Users.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -tags: - - PowerShell - - Reporting - - Scripting ---- - -``` powershell -InactiveDays = 30 -$Days = (Get-Date).Adddays(-($InactiveDays)) -Get-ADUser -Filter {LastLogonTimeStamp -lt $Days -and enabled -eq $true} -Properties LastLogonTimeStamp | -select-object Name,@{Name="Date"; Expression={[DateTime]::FromFileTime($_.lastLogonTimestamp).ToString('MM-dd-yyyy')}} -``` \ No newline at end of file diff --git a/scripts/Services/Email/Microsoft Exchange/Restart Exchange Services.md b/scripts/Services/Email/Microsoft Exchange/Restart Exchange Services.md deleted file mode 100644 index b3a5180..0000000 --- a/scripts/Services/Email/Microsoft Exchange/Restart Exchange Services.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -tags: - - Microsoft Exchange - - Email ---- - -### Purpose: -Sometimes Microsoft Exchange Server will misbehave and the services will need to be *bumped* to fix them. This script iterates over all of the Exchange-related services and restarts them automatically for you. - -``` powershell -$servicelist = Get-Service | Where-Object {$_.DisplayName -like "Microsoft Exchange *"} -$servicelist += Get-Service | Where-Object {$_.DisplayName -eq "IIS Admin Service"} -$servicelist += Get-Service | Where-Object { $_.DisplayName –eq "Windows Management Instrumentation" } -$servicelist += Get-Service | Where-Object { $_.DisplayName –eq "World Wide Web Publishing Service" } - -foreach($service in $servicelist){ - Set-Service $service -StartupType Automatic - Start-Service $service -} -``` \ No newline at end of file diff --git a/scripts/Powershell/Hyper V/Collapse Differencing Disk Chains.md b/scripts/Virtualization and Storage/Hyper-V/Collapse Differencing Disk Chains.md similarity index 94% rename from scripts/Powershell/Hyper V/Collapse Differencing Disk Chains.md rename to scripts/Virtualization and Storage/Hyper-V/Collapse Differencing Disk Chains.md index 48b38cf..82b5ca7 100644 --- a/scripts/Powershell/Hyper V/Collapse Differencing Disk Chains.md +++ b/scripts/Virtualization and Storage/Hyper-V/Collapse Differencing Disk Chains.md @@ -10,8 +10,9 @@ Sometimes things go awry with backup servers and Hyper-V and a bunch of extra `. This script automatically iterates through the entire differencing disk chain all the way back to the base disk / parent, and automatically collapses the chain downward from the newest checkpoint (provided as an argument to the script) to the original (non-differencing) base disk. This can automate a huge amount of work when this issue happens due to backup servers or other unexplainable anomalies. ## Powershell Script -You need to copy the contents of the following somewhere on your computer and save it as `Get-HyperVParentDisks.ps1`. -``` powershell +You need to copy the contents of the following somewhere on your computer and save it as `Get-HyperVParentDisks.ps1`. + +```powershell param ( [Parameter(Mandatory=$true, HelpMessage="Specify the path to the AVHDX file.")] [string]$AVHDXPath, @@ -36,11 +37,11 @@ function Get-AllParentDisksChain { $parentDiskChain = @() while ($CurrentDisk) { $parentDisk = Get-ParentDisk -ChildDisk $CurrentDisk - if ($parentDisk) { + if ($parentDisk) { $parentDiskChain += $CurrentDisk # Add the current disk to the chain before moving to the parent - $CurrentDisk = $parentDisk - } else { - break + $CurrentDisk = $parentDisk + } else { + break } } $parentDiskChain += $CurrentDisk # Add the base disk at the end of the chain @@ -49,7 +50,7 @@ function Get-AllParentDisksChain { function Merge-DiskIntoParent { param ([string]$ChildDisk, [string]$ParentDisk, [int]$DiskNumber, [int]$TotalDisks) - + if ($DryRun) { Write-Output "[Differential Disk $DiskNumber of $TotalDisks]" Write-Output "Child: $ChildDisk" @@ -100,7 +101,8 @@ Merge Disks: `.\Get-HyperVParentDisks.ps1 -MergeIntoParents -AVHDXPath "Z:\Example\Virtual Hard Disks\Example.avhdx"` !!! info "Example Output" - ``` + + ```text Starting parent disk search for: Z:\DISK-MERGE-TESTER\Virtual Hard Disks\DISK-MERGE-TESTER_E5F78673-3DAD-4211-AC0A-A3BDEB763B63.avhdx Total parent disks found: 6 @@ -136,7 +138,8 @@ Dry Run (Non-Destructive): `.\Get-HyperVParentDisks.ps1 -MergeIntoParents -DryRun -AVHDXPath "Z:\Example\Virtual Hard Disks\Example.avhdx"` !!! info "Example Output" - ``` + + ```text Starting parent disk search for: Z:\DISK-MERGE-TESTER\Virtual Hard Disks\DISK-MERGE-TESTER_E5F78673-3DAD-4211-AC0A-A3BDEB763B63.avhdx Total parent disks found: 6 @@ -145,7 +148,7 @@ Dry Run (Non-Destructive): Child: Z:\DISK-MERGE-TESTER\Virtual Hard Disks\DISK-MERGE-TESTER_E5F78673-3DAD-4211-AC0A-A3BDEB763B63.avhdx Parent: Z:\DISK-MERGE-TESTER\Virtual Hard Disks\DISK-MERGE-TESTER_8B9EDF27-6B7D-4766-AE60-ED67BF3055AE.avhdx [Dry Run] Would merge child into parent - + [Differential Disk 2 of 6] Child: Z:\DISK-MERGE-TESTER\Virtual Hard Disks\DISK-MERGE-TESTER_8B9EDF27-6B7D-4766-AE60-ED67BF3055AE.avhdx Parent: Z:\DISK-MERGE-TESTER\Virtual Hard Disks\DISK-MERGE-TESTER_6607B03C-E3F8-49CC-A69B-68BA3DACE81F.avhdx @@ -166,4 +169,7 @@ Dry Run (Non-Destructive): Parent: Z:\DISK-MERGE-TESTER\Virtual Hard Disks\DISK-MERGE-TESTER.vhdx [Dry Run] Would merge child into parent Merge process completed. - ``` \ No newline at end of file + ``` + +## Related Documentation +- [Related Virtualization and Storage Documentation](<../../../reference/Virtualization and Storage/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/Hyper V/Delete Locked VHDX File.md b/scripts/Virtualization and Storage/Hyper-V/Delete Locked VHDX File.md similarity index 67% rename from scripts/Powershell/Hyper V/Delete Locked VHDX File.md rename to scripts/Virtualization and Storage/Hyper-V/Delete Locked VHDX File.md index 0eca5e1..6d6a3c7 100644 --- a/scripts/Powershell/Hyper V/Delete Locked VHDX File.md +++ b/scripts/Virtualization and Storage/Hyper-V/Delete Locked VHDX File.md @@ -4,7 +4,7 @@ tags: - Scripting --- -**Purpose**: +## Purpose You may find that you cannot delete a VHDX file for a virtual machine you removed from Hyper-V and/or Hyper-V Failover Cluster, and either cannot afford to, or do not want to reboot your virtualization host(s) to unlock the file locked by `SYSTEM`. Run the following commands to unlock the file and delete it: @@ -12,4 +12,7 @@ Run the following commands to unlock the file and delete it: ```powershell Dismount-VHD -Path "C:\Path\To\Disk.vhdx" -ErrorAction SilentlyContinue Remove-Item -Path "C:\Path\To\Disk.vhdx" -Force -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Virtualization and Storage Documentation](<../../../reference/Virtualization and Storage/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/Hyper V/Failover Cluster/Force Reboot Remote Cluster Node.md b/scripts/Virtualization and Storage/Hyper-V/Failover Cluster/Force Reboot Remote Cluster Node.md similarity index 78% rename from scripts/Powershell/Hyper V/Failover Cluster/Force Reboot Remote Cluster Node.md rename to scripts/Virtualization and Storage/Hyper-V/Failover Cluster/Force Reboot Remote Cluster Node.md index e0788c8..0d74118 100644 --- a/scripts/Powershell/Hyper V/Failover Cluster/Force Reboot Remote Cluster Node.md +++ b/scripts/Virtualization and Storage/Hyper-V/Failover Cluster/Force Reboot Remote Cluster Node.md @@ -4,11 +4,12 @@ tags: - Scripting --- -**Purpose**: Sometimes a Hyper-V Failover Cluster node does not want to shut down, or is having issues preventing you from migrating VMs to another node in the cluster, etc. In these situations, you can run this script to force a cluster node to reboot itself. +## Purpose +Sometimes a Hyper-V Failover Cluster node does not want to shut down, or is having issues preventing you from migrating VMs to another node in the cluster, etc. In these situations, you can run this script to force a cluster node to reboot itself. !!! warning "Run from a Different Server" You absolutely do not want to run the script locally on the node that is having the issues. There are commands that can only take place if the script is ran on another node in the cluster (or another domain-joined device) logged-in with a domain administrator account. - + ```powershell # PowerShell Script to Reboot a Hyper-V Failover Cluster Node and Kill clussvc @@ -62,4 +63,7 @@ Invoke-Command -ComputerName $hostName -ScriptBlock { # Output the completion Write-Host "Reboot for $hostName should now be underway." -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Virtualization and Storage Documentation](<../../../../reference/Virtualization and Storage/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/Hyper V/Failover Cluster/Replication Bumper.md b/scripts/Virtualization and Storage/Hyper-V/Failover Cluster/Replication Bumper.md similarity index 88% rename from scripts/Powershell/Hyper V/Failover Cluster/Replication Bumper.md rename to scripts/Virtualization and Storage/Hyper-V/Failover Cluster/Replication Bumper.md index 2e75e7d..c9350f7 100644 --- a/scripts/Powershell/Hyper V/Failover Cluster/Replication Bumper.md +++ b/scripts/Virtualization and Storage/Hyper-V/Failover Cluster/Replication Bumper.md @@ -4,10 +4,10 @@ tags: - Scripting --- -**Purpose**: -This script *bumps* any replication that has entered a paused state due to a replication error. The script will record failed attempts at restarting the replication. The logs will rotate out every 5-days. +## Purpose +This script *bumps* any replication that has entered a paused state due to a replication error. The script will record failed attempts at restarting the replication. The logs will rotate out every 5-days. -``` powershell +```powershell # Define the directory to store the log files $logDir = "C:\ClusterStorage\Volume1\Scripts\Logs" if (-not (Test-Path $logDir)) { @@ -30,7 +30,7 @@ if (-not (Test-Path $logFile)) { } # Delete log files older than 5 days -Get-ChildItem -Path $logDir -Filter "ReplicationLog_*.txt" | Where-Object { +Get-ChildItem -Path $logDir -Filter "ReplicationLog_*.txt" | Where-Object { $_.CreationTime -lt (Get-Date).AddDays(-5) } | Remove-Item @@ -72,4 +72,7 @@ foreach ($node in $clusterNodes) { } } } -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Virtualization and Storage Documentation](<../../../../reference/Virtualization and Storage/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Virtualization and Storage/Linux/Grow an mdadm Array.md b/scripts/Virtualization and Storage/Linux/Grow an mdadm Array.md new file mode 100644 index 0000000..edc3e74 --- /dev/null +++ b/scripts/Virtualization and Storage/Linux/Grow an mdadm Array.md @@ -0,0 +1,20 @@ +--- +tags: + - RAID + - Bash + - Scripting + - Linux +--- + +## Purpose +Keep the recorded mdadm grow command and array-progress check together. Confirm the current array layout and intended RAID conversion before adapting the `/dev/md0` example. + +https://www.digitalocean.com/community/tutorials/how-to-create-raid-arrays-with-mdadm-on-ubuntu-16-04 + +```sh +sudo mdadm --grow /dev/md0 -l 5 +cat /proc/mdstat +``` + +## Related Documentation +- [Related Virtualization and Storage Documentation](<../../../reference/Virtualization and Storage/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Bash/ProxmoxVE/Deeplab Rollback Script.md b/scripts/Virtualization and Storage/Proxmox/Deeplab Rollback Script.md similarity index 84% rename from scripts/Bash/ProxmoxVE/Deeplab Rollback Script.md rename to scripts/Virtualization and Storage/Proxmox/Deeplab Rollback Script.md index c7e3a78..bbdc4d5 100644 --- a/scripts/Bash/ProxmoxVE/Deeplab Rollback Script.md +++ b/scripts/Virtualization and Storage/Proxmox/Deeplab Rollback Script.md @@ -10,7 +10,6 @@ tags: This script is ran via cronjob on `cluster-node-02` at midnight to rollback the deeplab environment automatically to a previous snapshot nightly. ### Bash Script - ```sh title="/root/deeplab-rollback.sh" #!/usr/bin/env bash # ProxmoxVE Nightly DeepLab Rollback Script @@ -58,12 +57,15 @@ Type `crontab -e` to add an entry to run the job at midnight every day. === "With Logging" - ``` sh + ```sh 0 0 * * * /root/deeplab-rollback.sh >> /var/log/deeplab-rollback.log 2>&1 ``` - + === "Without Logging" - ``` sh + ```sh 0 0 * * * /root/deeplab-rollback.sh 2>&1 - ``` \ No newline at end of file + ``` + +## Related Documentation +- [Related Proxmox Documentation](<../../../reference/Virtualization and Storage/Proxmox/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Virtualization and Storage/index.md b/scripts/Virtualization and Storage/index.md new file mode 100644 index 0000000..a0b8b8d --- /dev/null +++ b/scripts/Virtualization and Storage/index.md @@ -0,0 +1,18 @@ +--- +tags: + - Virtualization and Storage + - Scripts + - Documentation +--- + +# Virtualization and Storage +## Purpose +Find scripts for virtualization and storage. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- Hyper-V +- Linux +- Proxmox + +## Follow the Subject +[Virtualization and Storage](<../../reference/Virtualization and Storage/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/scripts/Powershell/General Purpose/Inactive User Profile Data Cleanup.md b/scripts/Windows and Linux/Windows/Clean Up Inactive User Profiles.md similarity index 98% rename from scripts/Powershell/General Purpose/Inactive User Profile Data Cleanup.md rename to scripts/Windows and Linux/Windows/Clean Up Inactive User Profiles.md index 29916cc..0511af4 100644 --- a/scripts/Powershell/General Purpose/Inactive User Profile Data Cleanup.md +++ b/scripts/Windows and Linux/Windows/Clean Up Inactive User Profiles.md @@ -19,8 +19,9 @@ This script is designed to iterate over every computer device within an Active D ### Script You can find the full script below, save it as `UserProfileDataPruner.ps1`: + ```powershell -<# +<# UserProfileDataPruner.ps1 Prune stale local user profile data on Windows workstations. @@ -328,4 +329,7 @@ process { } end { } -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Windows and Linux Documentation](<../../../reference/Windows and Linux/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/General Purpose/Fix Corrupted Windows Updates.md b/scripts/Windows and Linux/Windows/Repair Windows Update Components.md similarity index 79% rename from scripts/Powershell/General Purpose/Fix Corrupted Windows Updates.md rename to scripts/Windows and Linux/Windows/Repair Windows Update Components.md index ecd451a..7310481 100644 --- a/scripts/Powershell/General Purpose/Fix Corrupted Windows Updates.md +++ b/scripts/Windows and Linux/Windows/Repair Windows Update Components.md @@ -8,7 +8,7 @@ tags: ## Purpose Sometimes when you try to run Windows Updates, you may run into issues where updates just fail to install for seemingly nebulous reasons. You can run the following commands (in order) to try to resolve the issue. -!!! info "Run Commands from (CMD) Commandline, not powershell. +!!! info "Run Commands from (CMD) Commandline, not powershell." ```powershell # Imaging integrity Rrepair tools @@ -36,4 +36,7 @@ net start usosvc ``` !!! info "Attempt Windows Updates" - At this point, you can try re-running Windows Updates and seeing if the device makes it past the errors and installs the updates successfully or not. If not, **panic**. \ No newline at end of file + At this point, you can try re-running Windows Updates and seeing if the device makes it past the errors and installs the updates successfully or not. If not, **panic**. + +## Related Documentation +- [Related Windows and Linux Documentation](<../../../reference/Windows and Linux/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/General Purpose/Restart Service Domain Wide.md b/scripts/Windows and Linux/Windows/Start the RMM Agent Service Across the Domain.md similarity index 77% rename from scripts/Powershell/General Purpose/Restart Service Domain Wide.md rename to scripts/Windows and Linux/Windows/Start the RMM Agent Service Across the Domain.md index ee498bd..cec267f 100644 --- a/scripts/Powershell/General Purpose/Restart Service Domain Wide.md +++ b/scripts/Windows and Linux/Windows/Start the RMM Agent Service Across the Domain.md @@ -4,8 +4,8 @@ tags: - Scripting --- -**Purpose**: -Sometimes you need to restart a service across every computer in an Active Directory Domain. This powershell script will restart a specific service by name domain-wide. Each device will be processed in a serialized nature, one-by-one. +## Purpose +Start the `cagservice` RMM agent service on the domain servers selected by the script. The current implementation starts that service; it does not reboot computers or restart every service. !!! warning "Under Connstruction" This document is under construction and not generalized for general purpose use yet. Manual work needs to be done to repurpose this script for general usage. @@ -37,4 +37,7 @@ foreach ($server in $servers) { } Write-Host "Script execution completed." -ForegroundColor Green -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Windows and Linux Documentation](<../../../reference/Windows and Linux/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Powershell/General Purpose/Windows 11 Upgrade via UNC Path.md b/scripts/Windows and Linux/Windows/Upgrade Windows 11 from a UNC Path.md similarity index 99% rename from scripts/Powershell/General Purpose/Windows 11 Upgrade via UNC Path.md rename to scripts/Windows and Linux/Windows/Upgrade Windows 11 from a UNC Path.md index 8f8b49b..6eeb048 100644 --- a/scripts/Powershell/General Purpose/Windows 11 Upgrade via UNC Path.md +++ b/scripts/Windows and Linux/Windows/Upgrade Windows 11 from a UNC Path.md @@ -6,7 +6,7 @@ tags: - Scripting --- -**Purpose**: +## Purpose You may need to upgrade a device to Windows 11 using an ISO stored on a UNC Network Share, the script below handles that. !!! note "Environment Variables" @@ -23,7 +23,7 @@ You may need to upgrade a device to Windows 11 using an ISO stored on a UNC Netw ```powershell function generateSHA256 ($executable, $storedHash) { $fileBytes = [io.File]::ReadAllBytes("$executable") - $bytes = [Security.Cryptography.HashAlgorithm]::Create("SHA256").ComputeHash($fileBytes) + $bytes = [Security.Cryptography.HashAlgorithm]::Create("SHA256").ComputeHash($fileBytes) $varCalculatedHash=-Join ($bytes | ForEach {"{0:x2}" -f $_}) if ($storedHash -match $varCalculatedHash) { write-host "+ Filehash verified for file $executable`: $storedHash" @@ -239,7 +239,7 @@ if (($env:usrImagePath -as [string]).Length -lt 2 -or $env:usrImagePath -eq 'Sup write-host `r write-host " Generate a Windows 11 ISO download link good for 24 hours at:" write-host " https://www.microsoft.com/software-download/windows11" - exit 1 + exit 1 } elseif ($env:usrImagePath -match 'software-download.microsoft.com') { #microsoft write-host ": ISO Download location: Microsoft servers." @@ -372,7 +372,7 @@ switch -Regex ($varTPM -as [string]) { } '1$' { write-host "+ TPM installed and active." } $null { - write-host "! ERROR: A fault has occurred during the TPM checking subroutine. Please report this." + write-host "! ERROR: A fault has occurred during the TPM checking subroutine. Please report this." quitOr } @@ -516,4 +516,7 @@ if ($env:usrReboot -match 'true') { write-host " Please allow ~4 hours for the setup preparation step to conclude and then reboot the" write-host " device to begin the upgrade process." } -``` \ No newline at end of file +``` + +## Related Documentation +- [Related Windows and Linux Documentation](<../../../reference/Windows and Linux/index.md>) — Find the connected deployments, procedures, and references for this subject. diff --git a/scripts/Windows and Linux/index.md b/scripts/Windows and Linux/index.md new file mode 100644 index 0000000..e828cb8 --- /dev/null +++ b/scripts/Windows and Linux/index.md @@ -0,0 +1,16 @@ +--- +tags: + - Windows and Linux + - Scripts + - Documentation +--- + +# Windows and Linux +## Purpose +Find scripts for windows and linux. Follow the subject guide to choose the relevant environment and connect this material to the other document types. + +## Includes +- Windows + +## Follow the Subject +[Windows and Linux](<../../reference/Windows and Linux/index.md>) explains the relationships and offers starting points for the documented tasks. diff --git a/scripts/index.md b/scripts/index.md index 177b516..1788f76 100644 --- a/scripts/index.md +++ b/scripts/index.md @@ -1,34 +1,28 @@ --- tags: - Scripts - - Index - Documentation --- # Scripts ## Purpose -Quick-use scripts and snippets for day-to-day operations. +Find reusable utilities by the system or task they manage. Each page retains its language-specific code and operational context. ## Includes -- Bash, PowerShell, and Batch snippets -- One-off utilities and helpers +- Applications +- Automation +- Identity and Certificates +- Networking and Access +- Virtualization and Storage +- Windows and Linux -## New Document Template -````markdown -#