home-automation/README.md

Home Automation Monorepo

A Python-based home automation system built with Poetry in a monorepo structure. Features a microservices architecture with MQTT/Redis messaging, dynamic UI with realtime updates, and flexible device configuration.

Features

• Dynamic Dashboard: Responsive web UI with realtime device status via Server-Sent Events
• MQTT Integration: Device communication through MQTT broker with vendor abstraction
• Realtime Updates: Live device state updates via Redis Pub/Sub and SSE
• Flexible Layout: Configure rooms and device tiles via YAML
• Multiple Device Support: Lights with power and brightness control
• Clean Architecture: Separation of concerns with API-first design

Project Structure

home-automation/
├── apps/                    # Applications
│   ├── api/                # API Gateway (FastAPI, port 8001)
│   │   └── main.py        # REST API, SSE endpoint, device management
│   ├── abstraction/        # MQTT ↔ Redis Bridge
│   │   └── main.py        # Protocol translation layer
│   ├── rules/              # Rules Engine (planned)
│   └── ui/                 # Web Interface (FastAPI, port 8002)
│       ├── main.py        # Jinja2 templates, API client
│       ├── api_client.py  # HTTP client for API Gateway
│       ├── templates/     # HTML templates
│       │   ├── dashboard.html  # Dynamic dashboard
│       │   └── index.html      # Legacy static UI
│       └── static/        # CSS and assets
├── packages/               # Shared packages
│   └── home_capabilities/  # Core libraries
│       ├── light.py       # Light device models
│       └── layout.py      # UI layout models
├── config/                 # Configuration files
│   ├── devices.yaml       # Device definitions with features
│   └── layout.yaml        # UI room/device layout
├── tools/                  # Development tools
│   └── sim_test_lampe.py  # Multi-device MQTT simulator
├── infra/                  # Infrastructure
│   ├── docker-compose.yml
│   └── README.md
├── pyproject.toml          # Poetry configuration
├── PORTS.md               # Port allocation
└── README.md

Requirements

• Python 3.11+ (tested with 3.14.0)
• Poetry 2.2.1+
• MQTT Broker (e.g., Mosquitto)
• Redis Server

External Services

This system requires the following external services:

• MQTT Broker: 172.16.2.16:1883 (configured in config/devices.yaml)
• Redis Server: 172.23.1.116:6379/8 (configured in config/devices.yaml)

Setup

1. Install Poetry if you haven't already:

   curl -sSL https://install.python-poetry.org | python3 -
   

2. Install dependencies:

   poetry install
   

3. Activate the virtual environment:

   poetry shell
   

4. Configure devices and layout:

   • Edit config/devices.yaml for device definitions and MQTT/Redis settings
   • Edit config/layout.yaml for UI room organization

Configuration

devices.yaml

Defines available devices with their features and MQTT topics:

devices:
  - device_id: test_lampe_1
    type: light
    features:
      power: true
      brightness: true
    topics:
      set: "vendor/test_lampe_1/set"
      state: "vendor/test_lampe_1/state"

layout.yaml

Organizes devices into rooms for the UI:

rooms:
  - name: Wohnzimmer
    devices:
      - device_id: test_lampe_1
        title: Stehlampe
        icon: "LIGHT"
        rank: 10  # Lower rank = higher priority

Development

Dependencies

Key packages installed:

• Web Framework: FastAPI 0.120.3, Uvicorn 0.38.0
• Data Validation: Pydantic 2.12.3
• MQTT: aiomqtt 2.4.0 (async), paho-mqtt 2.1.0 (sync)
• Redis: redis 7.0.1
• HTTP Client: httpx 0.28.1
• Templates: Jinja2 3.1.6
• Config: PyYAML 6.0.3
• Testing: beautifulsoup4 4.14.2

Code Quality Tools

This project uses the following tools configured in pyproject.toml:

• Ruff: Fast Python linter
• Black: Code formatter
• Mypy: Static type checker

Run code quality checks:

# Format code
poetry run black .

# Lint code
poetry run ruff check .

# Type check
poetry run mypy .

Adding New Devices

1. Add device to config/devices.yaml:

   - device_id: new_device
     type: light
     features:
       power: true
       brightness: false
     topics:
       set: "vendor/new_device/set"
       state: "vendor/new_device/state"
   

2. Add device to rooms in config/layout.yaml:

   rooms:
     - name: Kitchen
       devices:
         - device_id: new_device
           title: Kitchen Light
           icon: "LIGHT"
           rank: 5
   

3. Restart API and UI services (they will auto-reload if using --reload)

4. Device will appear in dashboard automatically!

Extending Features

To add new device capabilities:

1. Update Pydantic models in packages/home_capabilities/
2. Add feature to devices.yaml
3. Extend dashboard template for UI controls
4. Update simulator or create new simulator for testing

Troubleshooting

Connection Issues

• SSE not connecting: Check API server is running on port 8001
• Device not responding: Check MQTT broker connectivity
• No updates in UI: Check abstraction layer and Redis connection

Check Logs

# API logs
tail -f /tmp/api.log

# Abstraction layer logs
tail -f /tmp/abstraction.log

# UI logs
tail -f /tmp/ui.log

Common Commands

# Check if services are running
ps aux | grep -E "uvicorn|abstraction"

# Check port usage
lsof -i :8001
lsof -i :8002

# Test MQTT connection
mosquitto_pub -h 172.16.2.16 -t test -m "hello"

Running Applications

Quick Start - All Services

Start all services in the background:

# 1. Start MQTT Abstraction Layer
poetry run python -m apps.abstraction.main > /tmp/abstraction.log 2>&1 &

# 2. Start API Gateway
poetry run uvicorn apps.api.main:app --host 0.0.0.0 --port 8001 > /tmp/api.log 2>&1 &

# 3. Start UI
poetry run uvicorn apps.ui.main:app --host 0.0.0.0 --port 8002 > /tmp/ui.log 2>&1 &

# 4. Start Device Simulator (optional)
poetry run python tools/sim_test_lampe.py &

Stop all services:

pkill -f "uvicorn apps" && pkill -f "apps.abstraction.main" && pkill -f "sim_test_lampe"

Port Configuration

See PORTS.md for detailed port allocation.

• API Server: http://localhost:8001
• UI Server: http://localhost:8002

API Server

Start the FastAPI server with auto-reload:

# Using uvicorn directly (port 8001)
poetry run uvicorn apps.api.main:app --reload --port 8001

# Or using the main function
poetry run python -m apps.api.main

The API will be available at:

• API Base: http://localhost:8001
• Interactive Docs: http://localhost:8001/docs
• OpenAPI Schema: http://localhost:8001/openapi.json

Available endpoints:

• GET /health - Health check endpoint
• GET /devices - List all devices with features
• GET /layout - Get UI layout configuration
• POST /devices/{device_id}/set - Control a device
• GET /realtime - Server-Sent Events stream for live updates

UI Server

Start the web interface:

# Using uvicorn directly (port 8002)
poetry run uvicorn apps.ui.main:app --reload --port 8002

# Or using the main function
poetry run python -m apps.ui.main

The UI will be available at:

• Dynamic Dashboard: http://localhost:8002/dashboard (or http://localhost:8002)
  • Realtime device status via SSE
  • Toggle buttons with state reflection
  • Brightness sliders for dimmable lights
  • Event log for all updates
  • Responsive layout from config/layout.yaml
• Legacy Static UI: http://localhost:8002/index.html
  • Fixed layout with test_lampe_1 and test_lampe_2

Abstraction Layer

The MQTT-Redis bridge translates between protocols:

poetry run python -m apps.abstraction.main

Functions:

• Subscribes to vendor-specific MQTT topics (vendor/*/state)
• Publishes state changes to Redis Pub/Sub (ui:updates)
• Enables decoupling of UI from MQTT

Device Simulator

Test your setup with the multi-device simulator:

poetry run python tools/sim_test_lampe.py

Simulates:

• test_lampe_1: Light with power and brightness control
• test_lampe_2: Simple light with power only
• test_lampe_3: Simple light with power only

The simulator:

• Subscribes to vendor/test_lampe_*/set topics
• Maintains device state (power, brightness)
• Publishes state to vendor/test_lampe_*/state (retained)
• Handles graceful shutdown (sets all lights to off)

Other Applications

# Rules Engine (planned)
poetry run python -m apps.rules.main

Architecture

Message Flow

User Action (UI) 
  → HTTP POST to API Gateway (/devices/{id}/set)
    → MQTT Publish (home/light/{id}/set)
      → Abstraction Layer receives
        → MQTT Publish (vendor/{id}/set)
          → Device/Simulator receives
            → Device State Update
              → MQTT Publish (vendor/{id}/state, retained)
                → Abstraction Layer receives
                  → Redis Pub/Sub (ui:updates)
                    → API Gateway /realtime SSE
                      → UI Updates (EventSource)

Key Components

1. API Gateway (apps/api/main.py)

   • Single source of truth for configuration
   • REST endpoints for device control
   • SSE endpoint for realtime updates
   • Reads devices.yaml and layout.yaml

2. UI (apps/ui/main.py)

   • Pure API consumer (no direct file access)
   • Fetches devices and layout via HTTP
   • Renders dynamic dashboard with Jinja2
   • Connects to SSE for live updates

3. Abstraction Layer (apps/abstraction/main.py)

   • Protocol translation (MQTT ↔ Redis)
   • Subscribes to vendor topics
   • Publishes to Redis for UI updates

4. Device Simulator (tools/sim_test_lampe.py)

   • Emulates physical devices
   • Responds to SET commands
   • Publishes STATE updates

Testing

Manual Testing

1. Start all services (see Quick Start above)

2. Open the dashboard: http://localhost:8002

3. Toggle a light and watch:

   • Button changes state (Einschalten ↔ Ausschalten)
   • Status updates in realtime
   • Event log shows all messages

4. Check MQTT traffic:

   # Subscribe to all topics
   mosquitto_sub -h 172.16.2.16 -t '#' -v
   
   # Publish test command
   mosquitto_pub -h 172.16.2.16 -t 'vendor/test_lampe_1/set' \
     -m '{"power":"on","brightness":75}'
   

5. Check Redis Pub/Sub:

   redis-cli -h 172.23.1.116 -p 6379 -n 8
   > SUBSCRIBE ui:updates
   

API Testing

# Get all devices
curl http://localhost:8001/devices | python3 -m json.tool

# Get layout
curl http://localhost:8001/layout | python3 -m json.tool

# Control a device
curl -X POST http://localhost:8001/devices/test_lampe_1/set \
  -H "Content-Type: application/json" \
  -d '{"type":"light","payload":{"power":"on"}}'

# Test SSE stream
curl -N http://localhost:8001/realtime

License

TBD