Files

194 lines
6.5 KiB
Bash
Raw Permalink Normal View History

2024-02-15 23:04:55 -05:00
#!/usr/bin/env bash
set -eEuo pipefail
2024-02-15 23:04:55 -05:00
function header_info() {
clear
cat <<"EOF"
_______ __ __ ______ _
/ ____(_) /__ _______ _______/ /____ ____ ___ /_ __/____(_)___ ___
/ /_ / / / _ \/ ___/ / / / ___/ __/ _ \/ __ `__ \ / / / ___/ / __ `__ \
/ __/ / / / __(__ ) /_/ (__ ) /_/ __/ / / / / / / / / / / / / / / / /
/_/ /_/_/\___/____/\__, /____/\__/\___/_/ /_/ /_/ /_/ /_/ /_/_/ /_/ /_/
/____/
EOF
}
BL="\033[36m"
RD="\033[01;31m"
GN="\033[1;92m"
CL="\033[m"
tools/pve: add data analytics / formatting / linting (#12034) * core: add progress; fix exit status Introduce post_progress_to_api() in alpine-install.func and install.func to send a lightweight, fire-and-forget telemetry ping (HTTP POST) that updates an existing telemetry record to "configuring" when DIAGNOSTICS=yes and RANDOM_UUID is set. The function is non-blocking (curl -m 5, errors ignored) and is invoked during container setup and after OS updates to signal active progress. Also adjust api_exit_script() in build.func to report success (post_update_to_api "done" "0") for cases where the script exited normally but a completion status wasn't posted, instead of reporting failure. * Safer tools.func load and improved error handling Replace process-substitution sourcing of tools.func with an explicit curl -> variable -> source via /dev/stdin, adding failure messages and a check that expected functions (e.g. fetch_and_deploy_gh_release) are present (misc/alpine-install.func, misc/install.func). Add categorize_error mapping for exit code 10 -> "config" (misc/api.func). Tweak build.func: minor pipeline formatting and change the ERR trap to capture the actual exit code and only call ensure_log_on_host/post_update on non-zero exits, preventing erroneous failure reporting. * tools: add data init and auto-reporting to tools and pve section Introduce telemetry helpers in misc/api.func: _telemetry_report_exit (reports success/failure via post_tool_to_api/post_addon_to_api) and init_tool_telemetry (reads DIAGNOSTICS, starts install timer and installs an EXIT trap to auto-report). Integrate telemetry into many tools/addon and tools/pve scripts by sourcing the remote api.func and calling init_tool_telemetry (guarded with declare -f). Also apply a minor arithmetic formatting tweak in misc/build.func for RECOVERY_ATTEMPT.
2026-02-17 16:36:20 +01:00
# Telemetry
source <(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/misc/api.func) 2>/dev/null || true
core: Execution ID & Telemetry Improvements (#12041) * fix: send telemetry BEFORE log collection in signal handlers - Swap ensure_log_on_host/post_update_to_api order in on_interrupt, on_terminate, api_exit_script, and inline SIGHUP/SIGINT/SIGTERM traps - For signal exits (>128): send telemetry immediately, then best-effort log collection - Add 2>/dev/null || true to all I/O in signal handlers to prevent SIGPIPE - Fix on_exit: exit_code=0 now reports 'done' instead of 'failed 1' - Root cause: pct pull hangs on dying containers blocked telemetry updates, leaving 595+ records stuck in 'installing' daily * feat: add execution_id to all telemetry payloads - Generate EXECUTION_ID from RANDOM_UUID in variables() - Export EXECUTION_ID to container environment - Add execution_id field to all 8 API payloads in api.func - Add execution_id to post_progress_to_api in install.func and alpine-install.func - Fallback to RANDOM_UUID when EXECUTION_ID not set (backward compat) * fix: correct telemetry type values for PVE and addon scripts - PVE scripts (tools/pve/*): change type 'tool' -> 'pve' - Addon scripts (tools/addon/*): fix 4 scripts that wrongly used 'tool' -> 'addon' (netdata, add-tailscale-lxc, add-netbird-lxc, all-templates) - api.func: post_tool_to_api sends type='pve', default fallback 'pve' - Aligns with PocketBase categories: lxc, vm, pve, addon * fix: persist diagnostics opt-in inside containers for addon telemetry - install.func + alpine-install.func: create /usr/local/community-scripts/diagnostics inside the container when DIAGNOSTICS=yes (from build.func export) - Enables addon scripts running later inside containers to find the opt-in - Update init_tool_telemetry default type from 'tool' to 'pve' * refactor: clean up diagnostics/telemetry opt-in system - diagnostics_check(): deduplicate heredoc (was 2x 22 lines), improve whiptail text with clear what/what-not collected, add telemetry + privacy links - diagnostics_menu(): better UX with current status, clear enable/disable buttons, note about existing containers - variables(): change DIAGNOSTICS default from 'yes' to 'no' (safe: no telemetry before user consents via diagnostics_check) - install.func + alpine-install.func: persist BOTH yes AND no in container so opt-out is explicit (not just missing file = no) - Fix typo 'menue' -> 'menu' in config file comments * fix: no pre-selection in telemetry dialog, link to telemetry-service README - Add --defaultno so 'No, opt out' is focused by default (user must Tab to Yes) - Change privacy link from discussions/1836 to telemetry-service#privacy--compliance * fix: use radiolist for telemetry dialog (no pre-selection) - Replace --yesno with --radiolist: user must actively SPACE-select an option - Both options start as OFF (no pre-selection) - Cancel/Exit defaults to 'no' (opt-out) * simplify: inline telemetry dialog text like other whiptail dialogs * improve: telemetry dialog with more detail, link to PRIVACY.md - Add what we collect / don't collect sections back to dialog - Link to telemetry-service/docs/PRIVACY.md instead of README anchor - Update config file comment with same link
2026-02-18 10:24:06 +01:00
declare -f init_tool_telemetry &>/dev/null && init_tool_telemetry "fstrim" "pve"
tools/pve: add data analytics / formatting / linting (#12034) * core: add progress; fix exit status Introduce post_progress_to_api() in alpine-install.func and install.func to send a lightweight, fire-and-forget telemetry ping (HTTP POST) that updates an existing telemetry record to "configuring" when DIAGNOSTICS=yes and RANDOM_UUID is set. The function is non-blocking (curl -m 5, errors ignored) and is invoked during container setup and after OS updates to signal active progress. Also adjust api_exit_script() in build.func to report success (post_update_to_api "done" "0") for cases where the script exited normally but a completion status wasn't posted, instead of reporting failure. * Safer tools.func load and improved error handling Replace process-substitution sourcing of tools.func with an explicit curl -> variable -> source via /dev/stdin, adding failure messages and a check that expected functions (e.g. fetch_and_deploy_gh_release) are present (misc/alpine-install.func, misc/install.func). Add categorize_error mapping for exit code 10 -> "config" (misc/api.func). Tweak build.func: minor pipeline formatting and change the ERR trap to capture the actual exit code and only call ensure_log_on_host/post_update on non-zero exits, preventing erroneous failure reporting. * tools: add data init and auto-reporting to tools and pve section Introduce telemetry helpers in misc/api.func: _telemetry_report_exit (reports success/failure via post_tool_to_api/post_addon_to_api) and init_tool_telemetry (reads DIAGNOSTICS, starts install timer and installs an EXIT trap to auto-report). Integrate telemetry into many tools/addon and tools/pve scripts by sourcing the remote api.func and calling init_tool_telemetry (guarded with declare -f). Also apply a minor arithmetic formatting tweak in misc/build.func for RECOVERY_ATTEMPT.
2026-02-17 16:36:20 +01:00
LOGFILE="/var/log/fstrim.log"
touch "$LOGFILE"
chmod 600 "$LOGFILE"
echo -e "\n----- $(date '+%Y-%m-%d %H:%M:%S') | fstrim Run by $(whoami) on $(hostname) -----" >>"$LOGFILE"
2024-02-15 23:04:55 -05:00
header_info
echo "Loading..."
whiptail --backtitle "Proxmox VE Helper Scripts" \
--title "About fstrim (LXC)" \
--msgbox "The 'fstrim' command releases unused blocks back to the storage device. This only makes sense for containers on SSD, NVMe, Thin-LVM, or storage with discard/TRIM support.\n\nIf your root filesystem or container disks are on classic HDDs, thick LVM, or unsupported storage types, running fstrim will have no effect.\n\nRecommended:\n- Use fstrim only on SSD, NVMe, or thin-provisioned storage with discard enabled.\n- For ZFS, ensure 'autotrim=on' is set on your pool.\n" 16 88
2024-02-15 23:04:55 -05:00
ROOT_FS=$(df -Th "/" | awk 'NR==2 {print $2}')
if [ "$ROOT_FS" != "ext4" ]; then
whiptail --backtitle "Proxmox VE Helper Scripts" \
--title "Warning" \
core: standardize exit codes and add mappings (#12467) * Standardize exit codes and add mappings Replace generic exit 1 usages with specific numeric exit codes and add corresponding explanations to the error lookup. This commit updates multiple misc/* scripts to return distinct codes for validation, Proxmox/LXC, networking, download and curl errors (e.g. 103-123, 64, 107-120, 206, 0 for explicit user cancels). It also updates curl error handling to propagate the original curl exit code and adds new entries in explain_exit_code and the error handler to improve diagnostics. * Set exit code 115 for update_os errors Change exit status from 6 to 115 in misc/alpine-install.func's update_os() error handlers when failing to download tools.func or when the expected functions are missing. This gives a distinct exit code for these specific failure cases. * Add tools/addon exit codes and use them Introduce exit codes 232-238 for Tools & Addon scripts in misc/api.func and misc/error_handler.func. Update addon scripts (tools/addon/adguardhome-sync.sh, tools/addon/copyparty.sh, tools/addon/cronmaster.sh) to return specific codes instead of generic exit 1: 238 for unsupported OS and 233 when the application is not installed/upgrade prerequisites are missing. This makes failures more descriptive and aligns scripts with the central error explanations. * Standardize exit codes in exporter addons Unify exit codes across exporter addon scripts: return 238 for unsupported OS detections and 233 when an update is requested but the exporter is not installed. Applied to nextcloud-exporter.sh, pihole-exporter.sh, prometheus-paperless-ngx-exporter.sh, and qbittorrent-exporter.sh to make failure modes distinguishable for callers/automation. * Use specific exit codes in addon scripts Replace generic exit 1 with distinct exit codes across multiple addon scripts to enable finer-grained error handling in automation. Exit codes introduced: 10 for Docker/Compose missing or user-declined Docker install, 233 for "nothing to update" cases, and 238 for unsupported OS cases. Affected files: tools/addon/arcane.sh, coolify.sh, dockge.sh, dokploy.sh, filebrowser-quantum.sh, filebrowser.sh, immich-public-proxy.sh, jellystat.sh, runtipi.sh. * Use specific exit codes in addon scripts Replace generic exit 1 with specific exit codes across multiple addon scripts to improve error signaling and handling. Files updated: tools/addon/add-netbird-lxc.sh (exit 238 on unsupported distro), tools/addon/add-tailscale-lxc.sh (treat user cancel as exit 0), tools/addon/glances.sh (exit 233 when not installed), tools/addon/komodo.sh (distinct exits for missing compose, legacy DB, backup/download failures, docker checks), tools/addon/netdata.sh (distinct exits for unsupported PVE versions, OS/codename detection, repo lookups), and tools/addon/phpmyadmin.sh (distinct exits for unsupported OS, network/download issues, package install/start failures, and invalid input). These changes make failures easier to identify and automate recovery or reporting. * Use specific exit codes in PVE scripts Replace generic exit 1 with distinct exit codes across tools/pve scripts to provide clearer failure signals for callers. post-pve-install.sh now returns 105 for unsupported Proxmox versions; pve-privilege-converter.sh uses 104 for non-root, 234 when no containers, and 235 for backup/conversion failures; update-apps.sh maps backup failures to 235, missing containers/selections to 234 (and UI cancellations to 0), missing backup storage to 119, and returns the actual container update exit code on failure. These changes improve diagnostics and allow external tooling to react to specific error conditions. * Standardize exit codes and behaviors Adjust exit codes and abort handling across multiple PVE helper scripts to provide clearer outcomes for automation and interactive flows. Changes include: - container-restore-from-backup.sh, core-restore-from-backup.sh: return 235 when no backups found (was 1). - fstrim.sh: treat user cancellation of non-ext4 warning as non-error (exit 0 instead of 1). - kernel-clean.sh: treat no selection or user abort as non-error (exit 0 instead of 1). - lxc-delete.sh: return 234 when no containers are present; treat no selection as non-error (exit 0). - nic-offloading-fix.sh: use specific non-zero codes for root check and tool install failures (exit 104, 237) and 236 when no matching interfaces (was 1). - pbs_microcode.sh, post-pmg-install.sh, post-pbs-install.sh: use distinct exit codes (232 and 105) for detected VM/PVE/unsupported distro conditions instead of generic 1. These modifications make scripts return distinct codes for different failure modes and ensure user-initiated aborts or benign conditions exit with 0 where appropriate. * Use exit 105 for unsupported PVE versions Standardize error handling by replacing generic exit 1 with exit 105 in pve_check() across multiple VM template scripts to indicate unsupported Proxmox VE versions. Also add API exit code 226 message for "Proxmox: VM disk import or post-creation setup failed" in misc/api.func. Affected files include misc/api.func and various vm/*-vm.sh scripts. * Use specific exit codes in VM scripts Replace generic exit 1 with distinct exit codes across vm/*.sh to make failures more actionable for callers. Changes include: use 226 for missing imported-disk references, 237 for pv installation failures, 115 for download/extract/ISO-related failures, 214 for insufficient disk space during FreeBSD decompression, and 119 for missing storage detection. Updated scripts: archlinux-vm.sh, docker-vm.sh, haos-vm.sh, openwrt-vm.sh, opnsense-vm.sh, truenas-vm.sh, umbrel-os-vm.sh.
2026-03-02 10:55:20 +01:00
--yesno "Root filesystem is not ext4 ($ROOT_FS).\nContinue anyway?" 12 80 || exit 0
2024-02-15 23:04:55 -05:00
fi
NODE=$(hostname)
EXCLUDE_MENU=()
STOPPED_MENU=()
MAX_NAME_LEN=0
MAX_STAT_LEN=0
# Build arrays with one pct list
mapfile -t CTLINES < <(pct list | awk 'NR>1')
for LINE in "${CTLINES[@]}"; do
CTID=$(awk '{print $1}' <<<"$LINE")
STATUS=$(awk '{print $2}' <<<"$LINE")
NAME=$(awk '{print $3}' <<<"$LINE")
((${#NAME} > MAX_NAME_LEN)) && MAX_NAME_LEN=${#NAME}
((${#STATUS} > MAX_STAT_LEN)) && MAX_STAT_LEN=${#STATUS}
done
FMT="%-${MAX_NAME_LEN}s | %-${MAX_STAT_LEN}s"
for LINE in "${CTLINES[@]}"; do
CTID=$(awk '{print $1}' <<<"$LINE")
STATUS=$(awk '{print $2}' <<<"$LINE")
NAME=$(awk '{print $3}' <<<"$LINE")
DESC=$(printf "$FMT" "$NAME" "$STATUS")
EXCLUDE_MENU+=("$CTID" "$DESC" "OFF")
if [[ "$STATUS" == "stopped" ]]; then
STOPPED_MENU+=("$CTID" "$DESC" "OFF")
fi
done
excluded_containers_raw=$(whiptail --backtitle "Proxmox VE Helper Scripts" \
--title "Containers on $NODE" \
--checklist "\nSelect containers to skip from trimming:\n" \
20 $((MAX_NAME_LEN + MAX_STAT_LEN + 20)) 12 "${EXCLUDE_MENU[@]}" 3>&1 1>&2 2>&3)
[ $? -ne 0 ] && exit
read -ra EXCLUDED <<<$(echo "$excluded_containers_raw" | tr -d '"')
TO_START=()
if [ ${#STOPPED_MENU[@]} -gt 0 ]; then
for ((i = 0; i < ${#STOPPED_MENU[@]}; i += 3)); do
CTID="${STOPPED_MENU[i]}"
DESC="${STOPPED_MENU[i + 1]}"
if [[ " ${EXCLUDED[*]} " =~ " $CTID " ]]; then
continue
fi
header_info
echo -e "${BL}[Info]${GN} Container $CTID ($DESC) is currently stopped.${CL}"
read -rp "Temporarily start for fstrim? [y/N]: " answer
if [[ "$answer" =~ ^[Yy]$ ]]; then
TO_START+=("$CTID")
fi
done
fi
declare -A WAS_STOPPED
for ct in "${TO_START[@]}"; do
WAS_STOPPED["$ct"]=1
done
2024-02-15 23:04:55 -05:00
function trim_container() {
local container="$1"
local name="$2"
2024-02-15 23:04:55 -05:00
header_info
2024-02-19 13:13:44 -05:00
echo -e "${BL}[Info]${GN} Trimming ${BL}$container${CL} \n"
local before_trim after_trim
local lv_name="vm-${container}-disk-0"
if lvs --noheadings -o lv_name 2>/dev/null | grep -qw "$lv_name"; then
before_trim=$(lvs --noheadings -o lv_name,data_percent 2>/dev/null | awk -v ctid="$lv_name" '$1 == ctid {gsub(/%/, "", $2); print $2}')
[[ -n "$before_trim" ]] && echo -e "${RD}Data before trim $before_trim%${CL}" || echo -e "${RD}Data before trim: not available${CL}"
else
before_trim=""
echo -e "${RD}Data before trim: not available (non-LVM storage)${CL}"
fi
local fstrim_output
fstrim_output=$(pct fstrim "$container" 2>&1)
if echo "$fstrim_output" | grep -qi "not supported"; then
echo -e "${RD}fstrim isnt supported on this storage!${CL}"
elif echo "$fstrim_output" | grep -Eq '([0-9]+(\.[0-9]+)?\s*[KMGT]?B)'; then
echo -e "${GN}fstrim result: $fstrim_output${CL}"
else
echo -e "${RD}fstrim result: $fstrim_output${CL}"
fi
if lvs --noheadings -o lv_name 2>/dev/null | grep -qw "$lv_name"; then
after_trim=$(lvs --noheadings -o lv_name,data_percent 2>/dev/null | awk -v ctid="$lv_name" '$1 == ctid {gsub(/%/, "", $2); print $2}')
[[ -n "$after_trim" ]] && echo -e "${GN}Data after trim $after_trim%${CL}" || echo -e "${GN}Data after trim: not available${CL}"
else
after_trim=""
echo -e "${GN}Data after trim: not available (non-LVM storage)${CL}"
fi
# Logging
echo "$(date '+%Y-%m-%d %H:%M:%S') | CTID=$container | Name=$name | Before=${before_trim:-N/A}% | After=${after_trim:-N/A}% | fstrim: $fstrim_output" >>"$LOGFILE"
sleep 0.5
2024-02-15 23:04:55 -05:00
}
for LINE in "${CTLINES[@]}"; do
CTID=$(awk '{print $1}' <<<"$LINE")
STATUS=$(awk '{print $2}' <<<"$LINE")
NAME=$(awk '{print $3}' <<<"$LINE")
if [[ " ${EXCLUDED[*]} " =~ " $CTID " ]]; then
2024-02-15 23:04:55 -05:00
header_info
echo -e "${BL}[Info]${GN} Skipping $CTID ($NAME, excluded)${CL}"
sleep 0.5
continue
fi
if pct config "$CTID" | grep -q "template:"; then
header_info
echo -e "${BL}[Info]${GN} Skipping $CTID ($NAME, template)${CL}\n"
sleep 0.5
continue
fi
if [[ "$STATUS" != "running" ]]; then
if [[ -n "${WAS_STOPPED[$CTID]:-}" ]]; then
2024-02-15 23:04:55 -05:00
header_info
echo -e "${BL}[Info]${GN} Starting $CTID ($NAME) for trim...${CL}"
pct start "$CTID"
sleep 2
else
header_info
echo -e "${BL}[Info]${GN} Skipping $CTID ($NAME, not running, not selected)${CL}"
sleep 0.5
2024-02-15 23:04:55 -05:00
continue
fi
fi
trim_container "$CTID" "$NAME"
if [[ -n "${WAS_STOPPED[$CTID]:-}" ]]; then
read -rp "Stop LXC $CTID ($NAME) again after trim? [Y/n]: " answer
if [[ ! "$answer" =~ ^[Nn]$ ]]; then
header_info
echo -e "${BL}[Info]${GN} Stopping $CTID ($NAME) again...${CL}"
pct stop "$CTID"
sleep 1
else
header_info
echo -e "${BL}[Info]${GN} Leaving $CTID ($NAME) running as requested.${CL}"
sleep 1
fi
2024-02-15 23:04:55 -05:00
fi
done
header_info
echo -e "${GN}Finished, LXC Containers Trimmed.${CL} \n"
echo -e "${BL}If you want to see the complete log: cat $LOGFILE${CL}"
exit 0