Merge pull request #14 from ShaYmez/copilot/add-pyproject-toml-file

Replace --break-system-packages with virtual environments for Debian 12+

CHANGELOG.md

@@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Changed
+- **BREAKING CHANGE - Modern Python Package Management**: For Debian 12+ (Bookworm/Trixie), HBMonv2 now uses a Python virtual environment instead of system-wide package installation
+- HBMonv2 Python dependencies installed in isolated virtual environment at `/opt/HBMonv2/venv` for Debian 12+
+- pip_install helper function now uses virtual environment for Debian 12+ (PEP 668 compliant)
+- systemd service file automatically configured to use virtual environment Python interpreter for Debian 12+
+- Added python3-venv to system dependencies for Debian 12+
 - Enhanced pip_install helper function with comprehensive error handling and user feedback
 - Added proper error checking after all pip package installations
 - Improved error messages to help users diagnose installation failures
@@ -14,10 +19,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Changed deprecated --force flag to --force-reinstall for attrs package installation
 
 ### Fixed
+- **Resolved externally-managed-environment errors on Debian 12/13** by using virtual environments instead of --break-system-packages flag
+- **Resolved package conflicts with system-installed cryptography** by isolating Python packages in virtual environment
 - Silent pip installation failures now properly reported to users
 - Missing requirements.txt file now handled gracefully with warning instead of silent failure
 - pip installation errors now cause the installer to exit with proper error messages
 
+### Technical Details
+- Debian 10-11: Continues to use standard system-wide pip installation (backward compatible)
+- Debian 12+: Uses modern virtual environment approach following PEP 668 standards
+- Virtual environment is automatically activated during installation
+- pip is upgraded within the virtual environment for latest features
+
 ## [1.5.0] - 2024-12-13
 
 ### Verified

README.md

@@ -3,7 +3,7 @@
 =======
 This is a multi-arch docker installer for HBlink3 and HBmonV2 combined for Debian 10, 11, 12, and 13 (Trixie). 
 
-**Note:** Debian 12 (Bookworm) and 13 (Trixie) support has been added with proper handling of pip installation restrictions and docker-compose compatibility. See [CHANGELOG.md](CHANGELOG.md) for version history and updates.
+**Note:** Debian 12 (Bookworm) and 13 (Trixie) support has been added with proper PEP 668 compliant Python package management using virtual environments, and docker-compose compatibility. HBMonv2 now runs in an isolated Python virtual environment on Debian 12+. See [CHANGELOG.md](CHANGELOG.md) for version history and updates.
 
 ![HBlink](img/HBLINK_logoV1.png "HBlink")
 
@@ -140,6 +140,18 @@ Within this installation includes the new HBMonv2 by Weldek SP2ONG
 * Includes lastheard database with auto cron installed	
 * Includes talkgroup html editable page	
 
+### Technical Details - Python Package Management
+
+**Debian 12+ (Bookworm/Trixie):** The installer uses modern Python package management following PEP 668 standards:
+- HBMonv2 runs in an isolated Python virtual environment at `/opt/HBMonv2/venv`
+- All Python dependencies are installed within this virtual environment, avoiding system-wide package conflicts
+- The systemd service automatically uses the virtual environment's Python interpreter
+- This approach eliminates "externally-managed-environment" errors and conflicts with system packages
+
+**Debian 10-11:** Standard pip installation to system Python is used for backward compatibility.
+
+This ensures clean, maintainable installations that follow modern Python best practices while maintaining compatibility with older Debian versions.
+
 ## Easy Installation And Upgrade
 The installation can be upgraded either by the use of a future scripts or by manually backing up your configuration and re-running the install script. Also the ability and really cool feature of docker-compose is that its easy to update the container with fresh images! Run by a simple command. Make sure you are in the /etc/hblink3 dir.
 ```sh

hblink3-docker-install.sh

@@ -76,7 +76,12 @@ install_docker_and_dependencies() {
         
         # Install base dependencies
         apt-get update
-        apt-get install -y $DEP
+        # For Debian 12+, add python3-venv to dependencies (PEP 668 compliance)
+        if [ $version -ge 12 ]; then
+                apt-get install -y $DEP python3-venv
+        else
+                apt-get install -y $DEP
+        fi
         sleep 2
         
         # Remove old Docker versions if present
@@ -234,20 +239,18 @@ sleep 2
 pip_install() {
         local args="$@"
         if [ $VERSION -ge 12 ]; then
-                # For Debian 12+, try with --break-system-packages flag first
+                # For Debian 12+, use virtual environment (PEP 668 compliant)
                 echo "Installing Python packages for Debian $VERSION: $args"
-                if pip3 install --break-system-packages $args; then
+                if [ -z "$VIRTUAL_ENV" ]; then
+                        echo "ERROR: Virtual environment not activated"
+                        return 1
+                fi
+                if pip3 install $args; then
                         echo "Successfully installed: $args"
                         return 0
                 else
-                        echo "Warning: Installation with --break-system-packages failed, trying standard installation..."
-                        if pip3 install $args; then
-                                echo "Successfully installed: $args"
-                                return 0
-                        else
-                                echo "ERROR: Failed to install: $args"
-                                return 1
-                        fi
+                        echo "ERROR: Failed to install: $args"
+                        return 1
                 fi
         else
                 # For Debian 10-11, use standard pip installation
@@ -265,6 +268,33 @@ pip_install() {
 echo "Installing Python dependencies..."
 cd $HBMONDIR
 
+# For Debian 12+, create and use a virtual environment (modern PEP 668 compliant approach)
+if [ $VERSION -ge 12 ]; then
+        echo "Creating Python virtual environment for Debian $VERSION..."
+        
+        # Create virtual environment
+        if [ ! -d "$HBMONDIR/venv" ]; then
+                python3 -m venv "$HBMONDIR/venv" || { echo "ERROR: Failed to create virtual environment"; exit 1; }
+                echo "Virtual environment created successfully at $HBMONDIR/venv"
+        else
+                echo "Virtual environment already exists at $HBMONDIR/venv"
+        fi
+        
+        # Activate virtual environment
+        source "$HBMONDIR/venv/bin/activate" || { echo "ERROR: Failed to activate virtual environment"; exit 1; }
+        # Verify activation by checking VIRTUAL_ENV is set
+        if [ -z "$VIRTUAL_ENV" ]; then
+                echo "ERROR: Virtual environment activation failed - VIRTUAL_ENV not set"
+                exit 1
+        fi
+        echo "Virtual environment activated"
+        
+        # Upgrade pip in the virtual environment
+        if ! pip3 install --upgrade pip; then
+                echo "WARNING: Failed to upgrade pip in virtual environment, continuing with existing version..."
+        fi
+fi
+
 # Install setuptools and wheel first
 if ! pip_install setuptools wheel; then
         echo "ERROR: Failed to install setuptools and wheel"
@@ -343,6 +373,28 @@ LOG_NAME        = 'hbmon.log'
 
 EOF
                 cp utils/hbmon.service /lib/systemd/system/
+                
+                # For Debian 12+, update the service file to use virtual environment
+                if [ $VERSION -ge 12 ]; then
+                        echo "Updating hbmon.service to use virtual environment..."
+                        # Update ExecStart to use venv Python (only if not already using venv)
+                        if ! grep -q "$HBMONDIR/venv/bin/python3" /lib/systemd/system/hbmon.service; then
+                                # Replace common Python interpreter paths with venv path
+                                sed -i "s|ExecStart=/usr/bin/python3|ExecStart=$HBMONDIR/venv/bin/python3|g" /lib/systemd/system/hbmon.service
+                                sed -i "s|ExecStart=python3 |ExecStart=$HBMONDIR/venv/bin/python3 |g" /lib/systemd/system/hbmon.service
+                                
+                                # Verify the service file was updated correctly
+                                if grep -q "ExecStart=$HBMONDIR/venv/bin/python3" /lib/systemd/system/hbmon.service; then
+                                        echo "Service file updated to use virtual environment"
+                                else
+                                        echo "WARNING: Service file update may not have completed correctly"
+                                        echo "Please manually verify /lib/systemd/system/hbmon.service uses $HBMONDIR/venv/bin/python3"
+                                fi
+                        else
+                                echo "Service file already configured to use virtual environment"
+                        fi
+                fi
+                
                 cp utils/lastheard /etc/cron.daily/
                 chmod +x /etc/cron.daily/lastheard
 echo ""