98481c5145
* Refactor GitHub Actions workflows for build, documentation, and linting - Consolidated build logic into reusable workflows (`task-build.yml` and `task-docs.yml`) to reduce duplication across multiple workflows. - Introduced `environments.json` to centralize the list of PlatformIO build environments, improving maintainability and clarity. - Updated `build.yml` and `build_and_docs_to_dev.yml` to utilize the new reusable workflows and environment definitions. - Enhanced `release.yml` to streamline the release process and integrate documentation generation. - Created reusable linting workflow (`task-lint.yml`) to standardize code formatting checks across the repository. - Simplified manual documentation workflow by leveraging the new reusable documentation workflow. - Improved artifact management and retention policies across workflows. - Updated dependencies and versions in workflows to ensure compatibility and performance. CI/CD pipeline agnostic of Workflow Engine and integrated on github actions - Implemented ci.sh for orchestrating the complete build pipeline. - Created ci_00_config.sh for centralized configuration of build scripts. - Created ci_build_firmware.sh for building firmware for specified PlatformIO environments. - Created ci_prepare_artifacts.sh for preparing firmware artifacts for upload or deployment. - Created ci_set_version.sh for updating version tags in firmware configuration files. - Created ci_build.sh to orchestrate the complete build pipeline. - Created ci_qa.sh for code linting and formatting checks using clang-format. - Created ci_site.sh for building and deploying VuePress documentation with version management. - Implemented checks for required tools and dependencies in the new scripts. - Improved internal scripts for better error handling and logging. UPDATE the web installer manifest generation and update documentation structure - Enhanced ci_list-env.sh to list environments from a JSON file. - Replaced common_wu.py and gen_wu.py scripts with new npm scripts for site generation and previewing on docsgen/gen_wu.js - Replaced generate_board_docs.py with docsgen/generated_board_docs.js - Added new npm scripts for integration of site generation on build phase. - Created preview_site.js to serve locally generated site over HTTPS with improved error handling. - Added new CI environments for CI builds in environments.json. - Deleted lint.yml as part of workflow cleanup. - Enhanced task-build.yml to include linting as a job and added support for specifying PlatformIO version. - Improved task-docs.yml to handle versioning more effectively and added clean option. Enhance documentation - ADD CLEAR Mark of development version of site - Updated README.md to include detailed workflow dependencies and relationships using mermaid diagrams. - Improved development.md with a quick checklist for contributors and clarified the code style guide. - Enhanced quick_start.md with tips for contributors and streamlined the workflow explanation. LINT FIX - Refined User_config.h for better formatting consistency. - Adjusted blufi.cpp and gatewayBT.cpp for improved code readability and consistency in formatting. - Updated gatewaySERIAL.cpp and mqttDiscovery.cpp to enhance logging error messages. - Improved sensorDS1820.cpp for better logging of device information. Add security scan workflows for vulnerability detection Add SBOM generation and upload to release workflow; update security scan summary handling Add shellcheck suppor + FIX shellcheck warning Enhance documentation for CI/CD scripts and workflows, adding details for security scanning and SBOM generation processes Fix formatting and alignment in BLE connection handling Reviewed the full web board presentation and the ESP32 web upload. The project uses a modern pattern where data is divided from the presentation layer. - Removed the `generate_board_docs` script. - Updated the `gen_wu` script in order to generate `boards-info.json`: the fail that containe all information about the configuration - Created and isolate the file `boards-info.js` to streamline the parsing of PlatformIO dependencies, modules, environments and improve the handling of library information. - Introduced vuepress component `BoardEnvironmentTable.vue` that render `boards-info.json` as UI card component - Introduced vuepress component `FlashEnvironmentSelector.vue` that render a selectred environment from `boards-info.json` and provide esp-web-upload feature on it - Introduced a new board page `board-selector.md` for improved firmware selection. - Updated `web-install.md` to enhance the firmware upload process, including a new board environment table. - Enhanced custom descriptions in `environments.ini` to include HTML links for better user guidance and board image link Add CC1101 initialization improvements and logging enhancements Add installation step for PlatformIO dependencies in documentation workflow Remove ci_set_version.sh script and associated versioning functionality * Fix comment provisined Fix PlatformIO version input reference in documentation workflow Remove outdated Squeezelite-ESP32 installer documentation
7.1 KiB
Upload
OpenMQTTGateway provides several installation approaches suited for different use cases. The quickest path uses pre-built binaries that work with standard configurations. For custom parameters or specific gateway module combinations, you can build firmware using your development environment. Alternatively, you can flash devices directly from your web browser without installing any software.
Choosing Your Upload Method
Web-Based Installation
The web installation method represents the easiest way to get started with OpenMQTTGateway. You can flash your device directly from your browser without downloading files or installing development tools. This method works on Windows, macOS, and Linux with Chrome, Edge, or Opera browsers.
The web installer displays available environments for different boards and gateway configurations. Select your board type, connect via USB, and click the connect button. The installer handles everything automatically, including erasing old firmware and writing the new one. The web installer uses ESP Web Tools technology to communicate directly with your ESP device.
Ready-to-Go Binary Installation
Downloading and installing pre-built binaries offers control over the flashing process using desktop tools. Download the binary files for your board from the GitHub releases page.
For ESP32 devices, you need the firmware binary, bootloader, and boot application partition files written to specific memory addresses. Windows users can use the ESP32 Flash Download Tool from Espressif. On Linux and macOS, the esptool.py command-line utility provides a straightforward upload method.
This method works well for standard configurations without modifying source code. After flashing, you can still configure WiFi, MQTT broker settings, and basic parameters through the configuration portal.
Custom Build and Upload
Building from source becomes necessary when you need specific pin assignments, custom MQTT topics, or particular gateway module combinations not available in pre-built binaries.
PlatformIO provides the recommended build environment. After downloading the source code from GitHub, you will find a platformio.ini file defining build environments for various hardware combinations.
The configuration system uses a layering approach where default values from User_config.h and config_XX.h files can be overridden by build flags in platformio.ini or environments.ini. You can embed WiFi credentials and MQTT settings at build time for automatic connection on first boot.
Browser-Based Building with Gitpod
For those who want to build custom firmware without setting up a local development environment, Gitpod offers a cloud-based solution. By clicking on the Gitpod link, you get a complete development environment running in your browser with PlatformIO already installed and configured.
After the automatic initial build completes, modify the environment configuration by editing environments.ini and run build commands in the browser terminal. Download the generated firmware files and flash them using the binary installation method.
Configuring Network and MQTT Settings
After flashing firmware, configure network connectivity and MQTT broker settings using either runtime or build-time approaches.
Runtime Configuration Portal
When you power on a freshly flashed device, it creates a WiFi access point named OpenMQTTGateway or starting with OMG_. You can find detailed information about the configuration portal here.
Connecting to this access point opens a portal where you configure your WiFi network, MQTT broker details, and optional security settings including TLS encryption and certificates. For devices with Ethernet, access the portal through the LAN IP address and configure WiFi as fallback connectivity.
The portal accepts broker IP addresses or hostnames with mDNS support like homeassistant.local. Set a gateway password to protect future configuration changes, OTA updates, and web interface access.
Build-Time Configuration
Alternatively, embed network and MQTT settings directly in firmware during the build process. Set parameters in User_config.h or add them as build flags in your PlatformIO environment definition. Store sensitive information in a separate _env.ini file excluded from version control.
Advanced Configuration Options
Beyond basic connectivity, OpenMQTTGateway supports several advanced features that enhance security and integration capabilities.
Secure MQTT Connections
For deployments over the internet or public networks, enable TLS encryption to secure communication between the gateway and MQTT broker. Configure your broker with a valid certificate and obtain the Certificate Authority certificate. The gateway can verify server identity against this certificate or connect with encryption without validation.
Provide the CA certificate at build time in default_server_cert.h or paste it into the configuration portal. The gateway supports both self-signed certificates and those from public certificate authorities.
Home Assistant Auto-Discovery
When you use Home Assistant as your home automation platform, OpenMQTTGateway automatically creates device entries and sensors through Home Assistant's MQTT discovery protocol, enabled by default in all standard builds.
Enable discovery in your Home Assistant MQTT integration settings and create a dedicated MQTT user. The gateway registers itself as a device and creates sensor entities automatically, appearing in Configuration → Devices section.
Topic Customization
The gateway publishes messages to MQTT topics following the format home/OpenMQTTGateway/GATEWAYtoMQTT. Enable the valueAsATopic feature to append received values to the topic path, making topic-based filtering easier and avoiding warnings in certain controllers.
Next Steps
After successfully uploading firmware and configuring your gateway, you can proceed to configure specific gateway modules for RF, IR, Bluetooth, LoRa, or other protocols you want to use. Each module has configuration options adjustable through MQTT commands or the web interface without rebuilding firmware.
The troubleshooting section covers common issues, but if you encounter problems not addressed here, the OpenMQTTGateway community forum provides an active place to ask questions and share solutions with other users.