diff --git a/README.md b/README.md index 685dae0..fabd07b 100644 --- a/README.md +++ b/README.md @@ -1,29 +1,63 @@ # Home Automation Monorepo -A Python-based home automation system built with Poetry in a monorepo structure. +A Python-based home automation system built with Poetry in a monorepo structure. Features a microservices architecture with MQTT/Redis messaging, dynamic UI with realtime updates, and flexible device configuration. + +## Features + +- **Dynamic Dashboard**: Responsive web UI with realtime device status via Server-Sent Events +- **MQTT Integration**: Device communication through MQTT broker with vendor abstraction +- **Realtime Updates**: Live device state updates via Redis Pub/Sub and SSE +- **Flexible Layout**: Configure rooms and device tiles via YAML +- **Multiple Device Support**: Lights with power and brightness control +- **Clean Architecture**: Separation of concerns with API-first design ## Project Structure ``` home-automation/ ├── apps/ # Applications -│ ├── api/ # API service -│ ├── abstraction/ # Abstraction layer -│ ├── rules/ # Rules engine -│ └── ui/ # User interface +│ ├── api/ # API Gateway (FastAPI, port 8001) +│ │ └── main.py # REST API, SSE endpoint, device management +│ ├── abstraction/ # MQTT ↔ Redis Bridge +│ │ └── main.py # Protocol translation layer +│ ├── rules/ # Rules Engine (planned) +│ └── ui/ # Web Interface (FastAPI, port 8002) +│ ├── main.py # Jinja2 templates, API client +│ ├── api_client.py # HTTP client for API Gateway +│ ├── templates/ # HTML templates +│ │ ├── dashboard.html # Dynamic dashboard +│ │ └── index.html # Legacy static UI +│ └── static/ # CSS and assets ├── packages/ # Shared packages -│ └── home_capabilities/ # Home capabilities library +│ └── home_capabilities/ # Core libraries +│ ├── light.py # Light device models +│ └── layout.py # UI layout models +├── config/ # Configuration files +│ ├── devices.yaml # Device definitions with features +│ └── layout.yaml # UI room/device layout +├── tools/ # Development tools +│ └── sim_test_lampe.py # Multi-device MQTT simulator ├── infra/ # Infrastructure │ ├── docker-compose.yml │ └── README.md ├── pyproject.toml # Poetry configuration +├── PORTS.md # Port allocation └── README.md ``` ## Requirements -- Python 3.11+ -- Poetry +- Python 3.11+ (tested with 3.14.0) +- Poetry 2.2.1+ +- MQTT Broker (e.g., Mosquitto) +- Redis Server + +### External Services + +This system requires the following external services: + +- **MQTT Broker**: `172.16.2.16:1883` (configured in `config/devices.yaml`) +- **Redis Server**: `172.23.1.116:6379/8` (configured in `config/devices.yaml`) ## Setup @@ -42,8 +76,57 @@ home-automation/ poetry shell ``` +4. Configure devices and layout: + - Edit `config/devices.yaml` for device definitions and MQTT/Redis settings + - Edit `config/layout.yaml` for UI room organization + +## Configuration + +### devices.yaml + +Defines available devices with their features and MQTT topics: + +```yaml +devices: + - device_id: test_lampe_1 + type: light + features: + power: true + brightness: true + topics: + set: "vendor/test_lampe_1/set" + state: "vendor/test_lampe_1/state" +``` + +### layout.yaml + +Organizes devices into rooms for the UI: + +```yaml +rooms: + - name: Wohnzimmer + devices: + - device_id: test_lampe_1 + title: Stehlampe + icon: "LIGHT" + rank: 10 # Lower rank = higher priority +``` + ## Development +### Dependencies + +Key packages installed: + +- **Web Framework**: FastAPI 0.120.3, Uvicorn 0.38.0 +- **Data Validation**: Pydantic 2.12.3 +- **MQTT**: aiomqtt 2.4.0 (async), paho-mqtt 2.1.0 (sync) +- **Redis**: redis 7.0.1 +- **HTTP Client**: httpx 0.28.1 +- **Templates**: Jinja2 3.1.6 +- **Config**: PyYAML 6.0.3 +- **Testing**: beautifulsoup4 4.14.2 + ### Code Quality Tools This project uses the following tools configured in `pyproject.toml`: @@ -65,8 +148,105 @@ poetry run ruff check . poetry run mypy . ``` +### Adding New Devices + +1. Add device to `config/devices.yaml`: + ```yaml + - device_id: new_device + type: light + features: + power: true + brightness: false + topics: + set: "vendor/new_device/set" + state: "vendor/new_device/state" + ``` + +2. Add device to rooms in `config/layout.yaml`: + ```yaml + rooms: + - name: Kitchen + devices: + - device_id: new_device + title: Kitchen Light + icon: "LIGHT" + rank: 5 + ``` + +3. Restart API and UI services (they will auto-reload if using `--reload`) + +4. Device will appear in dashboard automatically! + +### Extending Features + +To add new device capabilities: + +1. Update Pydantic models in `packages/home_capabilities/` +2. Add feature to `devices.yaml` +3. Extend dashboard template for UI controls +4. Update simulator or create new simulator for testing + +## Troubleshooting + +### Connection Issues + +- **SSE not connecting**: Check API server is running on port 8001 +- **Device not responding**: Check MQTT broker connectivity +- **No updates in UI**: Check abstraction layer and Redis connection + +### Check Logs + +```bash +# API logs +tail -f /tmp/api.log + +# Abstraction layer logs +tail -f /tmp/abstraction.log + +# UI logs +tail -f /tmp/ui.log +``` + +### Common Commands + +```bash +# Check if services are running +ps aux | grep -E "uvicorn|abstraction" + +# Check port usage +lsof -i :8001 +lsof -i :8002 + +# Test MQTT connection +mosquitto_pub -h 172.16.2.16 -t test -m "hello" +``` + ### Running Applications +#### Quick Start - All Services + +Start all services in the background: + +```bash +# 1. Start MQTT Abstraction Layer +poetry run python -m apps.abstraction.main > /tmp/abstraction.log 2>&1 & + +# 2. Start API Gateway +poetry run uvicorn apps.api.main:app --host 0.0.0.0 --port 8001 > /tmp/api.log 2>&1 & + +# 3. Start UI +poetry run uvicorn apps.ui.main:app --host 0.0.0.0 --port 8002 > /tmp/ui.log 2>&1 & + +# 4. Start Device Simulator (optional) +poetry run python tools/sim_test_lampe.py & +``` + +Stop all services: + +```bash +pkill -f "uvicorn apps" && pkill -f "apps.abstraction.main" && pkill -f "sim_test_lampe" +``` + #### Port Configuration See `PORTS.md` for detailed port allocation. @@ -93,7 +273,10 @@ The API will be available at: Available endpoints: - `GET /health` - Health check endpoint -- `GET /spec` - Capabilities specification +- `GET /devices` - List all devices with features +- `GET /layout` - Get UI layout configuration +- `POST /devices/{device_id}/set` - Control a device +- `GET /realtime` - Server-Sent Events stream for live updates #### UI Server @@ -108,20 +291,142 @@ poetry run python -m apps.ui.main ``` The UI will be available at: -- Main page: http://localhost:8002 -- `GET /spec` - Capabilities specification +- **Dynamic Dashboard**: http://localhost:8002/dashboard (or http://localhost:8002) + - Realtime device status via SSE + - Toggle buttons with state reflection + - Brightness sliders for dimmable lights + - Event log for all updates + - Responsive layout from `config/layout.yaml` +- **Legacy Static UI**: http://localhost:8002/index.html + - Fixed layout with test_lampe_1 and test_lampe_2 + +#### Abstraction Layer + +The MQTT-Redis bridge translates between protocols: + +```bash +poetry run python -m apps.abstraction.main +``` + +Functions: +- Subscribes to vendor-specific MQTT topics (`vendor/*/state`) +- Publishes state changes to Redis Pub/Sub (`ui:updates`) +- Enables decoupling of UI from MQTT + +#### Device Simulator + +Test your setup with the multi-device simulator: + +```bash +poetry run python tools/sim_test_lampe.py +``` + +Simulates: +- `test_lampe_1`: Light with power and brightness control +- `test_lampe_2`: Simple light with power only +- `test_lampe_3`: Simple light with power only + +The simulator: +- Subscribes to `vendor/test_lampe_*/set` topics +- Maintains device state (power, brightness) +- Publishes state to `vendor/test_lampe_*/state` (retained) +- Handles graceful shutdown (sets all lights to off) #### Other Applications ```bash -# Abstraction -poetry run python -m apps.abstraction.main - -# Rules +# Rules Engine (planned) poetry run python -m apps.rules.main +``` -# UI -poetry run python -m apps.ui.main +## Architecture + +### Message Flow + +``` +User Action (UI) + → HTTP POST to API Gateway (/devices/{id}/set) + → MQTT Publish (home/light/{id}/set) + → Abstraction Layer receives + → MQTT Publish (vendor/{id}/set) + → Device/Simulator receives + → Device State Update + → MQTT Publish (vendor/{id}/state, retained) + → Abstraction Layer receives + → Redis Pub/Sub (ui:updates) + → API Gateway /realtime SSE + → UI Updates (EventSource) +``` + +### Key Components + +1. **API Gateway** (`apps/api/main.py`) + - Single source of truth for configuration + - REST endpoints for device control + - SSE endpoint for realtime updates + - Reads `devices.yaml` and `layout.yaml` + +2. **UI** (`apps/ui/main.py`) + - Pure API consumer (no direct file access) + - Fetches devices and layout via HTTP + - Renders dynamic dashboard with Jinja2 + - Connects to SSE for live updates + +3. **Abstraction Layer** (`apps/abstraction/main.py`) + - Protocol translation (MQTT ↔ Redis) + - Subscribes to vendor topics + - Publishes to Redis for UI updates + +4. **Device Simulator** (`tools/sim_test_lampe.py`) + - Emulates physical devices + - Responds to SET commands + - Publishes STATE updates + +## Testing + +### Manual Testing + +1. Start all services (see Quick Start above) + +2. Open the dashboard: http://localhost:8002 + +3. Toggle a light and watch: + - Button changes state (Einschalten ↔ Ausschalten) + - Status updates in realtime + - Event log shows all messages + +4. Check MQTT traffic: + ```bash + # Subscribe to all topics + mosquitto_sub -h 172.16.2.16 -t '#' -v + + # Publish test command + mosquitto_pub -h 172.16.2.16 -t 'vendor/test_lampe_1/set' \ + -m '{"power":"on","brightness":75}' + ``` + +5. Check Redis Pub/Sub: + ```bash + redis-cli -h 172.23.1.116 -p 6379 -n 8 + > SUBSCRIBE ui:updates + ``` + +### API Testing + +```bash +# Get all devices +curl http://localhost:8001/devices | python3 -m json.tool + +# Get layout +curl http://localhost:8001/layout | python3 -m json.tool + +# Control a device +curl -X POST http://localhost:8001/devices/test_lampe_1/set \ + -H "Content-Type: application/json" \ + -d '{"type":"light","payload":{"power":"on"}}' + +# Test SSE stream +curl -N http://localhost:8001/realtime ``` ## License diff --git a/apps/ui/templates/index.html b/apps/ui/templates/index.html index 338f97b..8c6c019 100644 --- a/apps/ui/templates/index.html +++ b/apps/ui/templates/index.html @@ -187,7 +187,7 @@
Realtime Status: Verbinde...
Warte auf Events...