diff --git a/docs/prerequisites/board.md b/docs/prerequisites/board.md
index 3ece66cb..71cec487 100644
--- a/docs/prerequisites/board.md
+++ b/docs/prerequisites/board.md
@@ -1,3 +1,7 @@
+---
+pageClass: table-generated-page
+---
+
# Boards
OpenMQTTGateway is not closed to one board or type of board, by using the power of the Arduino framework and libraries that are cross compatibles it let you many choice of hardware, from an ESP8266 to an ESP32.
@@ -43,4 +47,10 @@ The plug is available in North America only, other regions are planned.
Choosing your board depends heavily on the technologies you want to use with it.
To have a good overview of the compatibilities per board you can refer to the compatible modules attributes of each [board](https://compatible.openmqttgateway.com/index.php/boards/).
-The choice between these boards will depend on your knowledge and your requirements in terms of reliability, situation, modules wanted and devices you already have. The table below present those (auto-generated)
+The choice between these boards will depend on your knowledge and your requirements in terms of reliability, situation, modules wanted and devices you already have. Use the table below to explore the latest environments.
+
+
+
diff --git a/docs/prerequisites/boards/.gitignore b/docs/prerequisites/boards/.gitignore
deleted file mode 100644
index 86d0cb27..00000000
--- a/docs/prerequisites/boards/.gitignore
+++ /dev/null
@@ -1,4 +0,0 @@
-# Ignore everything in this directory
-*
-# Except this file
-!.gitignore
\ No newline at end of file
diff --git a/docs/setitup/rf.md b/docs/setitup/rf.md
index e6f3c7bb..0c44fdb5 100644
--- a/docs/setitup/rf.md
+++ b/docs/setitup/rf.md
@@ -51,8 +51,22 @@ With SRX882S receiver connect the CS pin to 3.3V
|ESP8266|D2/**D3**/D1/D8|**RX**/D2|D5|**3V3**|D7|D6|D8|GND
|ESP32|**D27**|D12|D18|**3V3**|D23|D19|D5|GND
-To use the CC1101 module, `ZradioCC1101` must be uncomment in the `User_config.h` or added to the `build_flags`.
-More information about the [CC1101 wiring](https://github.com/LSatan/SmartRC-CC1101-Driver-Lib#wiring). ( Please note that with OMG we are recommending CC1101 GDO2 to be connected to ESP32 D27 and GDO0 to be connected to D12, this is different than the LSatan diagram. This is due to the ESP32 using D2 as part of the boot process. )
+To use the CC1101 module, `ZradioCC1101` must be uncomment in the `User_config.h` or added to the `build_flags`.
+
+More information about the [CC1101 wiring](https://github.com/LSatan/SmartRC-CC1101-Driver-Lib#wiring).
+
+
+:::tip Please note that with OMG we are recommending CC1101 GDO2 to be connected to ESP32 D27 and GDO0 to be connected to D12, this is different than the LSatan diagram. This is due to the ESP32 using D2 as part of the boot process.
+
+If you want to use custom SPI pins for the CC1101 module, you can define the following variables in your `User_config.h` or as `build_flags` in `platformio.ini`:
+ - `RF_CC1101_SCK`: SPI clock pin (SCK)
+ - `RF_CC1101_SCK`: SPI clock pin (SCK)
+ - `RF_CC1101_MISO`: SPI MISO pin (Master In Slave Out)
+ - `RF_CC1101_MOSI`: SPI MOSI pin (Master Out Slave In)
+ - `RF_CC1101_CS`: SPI chip select pin (CSN)
+
+When **all** these variables are defined, OpenMQTTGateway will use your custom pinout for the CC1101 connection. This is useful if your board does not use the default pins or if you want to avoid conflicts with other devices.
+:::
## ESP32 Hardware setup
@@ -73,3 +87,5 @@ The RF processing can be achieved after the modification by either RF, RF2 or Pi
## WIFI RF GATEWAY Hardware setup
This board doesn't require any hardware modifications.
+
+
diff --git a/docs/upload/ReadMe.md b/docs/upload/ReadMe.md
new file mode 100644
index 00000000..7ec95e21
--- /dev/null
+++ b/docs/upload/ReadMe.md
@@ -0,0 +1,79 @@
+# Upload
+
+OpenMQTTGateway provides several installation approaches suited for different use cases. The quickest path uses pre-built binaries that work with standard configurations. For custom parameters or specific gateway module combinations, you can build firmware using your development environment. Alternatively, you can flash devices directly from your web browser without installing any software.
+
+
+ 
+
+
+## Choosing Your Upload Method
+
+### Web-Based Installation
+
+The [web installation](web-install.md) method represents the easiest way to get started with OpenMQTTGateway. You can flash your device directly from your browser without downloading files or installing development tools. This method works on Windows, macOS, and Linux with Chrome, Edge, or Opera browsers.
+
+The web installer displays available environments for different boards and gateway configurations. Select your board type, connect via USB, and click the connect button. The installer handles everything automatically, including erasing old firmware and writing the new one. The web installer uses [ESP Web Tools](https://esphome.github.io/esp-web-tools/) technology to communicate directly with your ESP device.
+
+### Ready-to-Go Binary Installation
+
+[Downloading and installing pre-built binaries](binaries.md) offers control over the flashing process using desktop tools. Download the binary files for your board from the [GitHub releases page](https://github.com/1technophile/OpenMQTTGateway/releases).
+
+For ESP32 devices, you need the firmware binary, bootloader, and boot application partition files written to specific memory addresses. Windows users can use the ESP32 Flash Download Tool from Espressif. On Linux and macOS, the esptool.py command-line utility provides a straightforward upload method.
+
+This method works well for standard configurations without modifying source code. After flashing, you can still configure WiFi, MQTT broker settings, and basic parameters through the configuration portal.
+
+### Custom Build and Upload
+
+[Building from source](builds.md) becomes necessary when you need specific pin assignments, custom MQTT topics, or particular gateway module combinations not available in pre-built binaries.
+
+[PlatformIO](https://platformio.org/) provides the recommended build environment. After downloading the [source code from GitHub](https://github.com/1technophile/OpenMQTTGateway), you will find a `platformio.ini` file defining build environments for various hardware combinations.
+
+The configuration system uses a layering approach where default values from [`User_config.h`](https://github.com/1technophile/OpenMQTTGateway/blob/development/main/User_config.h) and [`config_XX.h`](https://github.com/1technophile/OpenMQTTGateway/tree/development/main) files can be overridden by build flags in `platformio.ini` or `environments.ini`. You can embed WiFi credentials and MQTT settings at build time for automatic connection on first boot.
+
+### Browser-Based Building with Gitpod
+
+For those who want to build custom firmware without setting up a local development environment, [Gitpod](gitpod.md) offers a cloud-based solution. By clicking on the [Gitpod link](https://gitpod.io#https://github.com/1technophile/OpenMQTTGateway/tree/development), you get a complete development environment running in your browser with PlatformIO already installed and configured.
+
+After the automatic initial build completes, modify the environment configuration by editing `environments.ini` and run build commands in the browser terminal. Download the generated firmware files and flash them using the binary installation method.
+
+## Configuring Network and MQTT Settings
+
+After flashing firmware, configure network connectivity and MQTT broker settings using either runtime or build-time approaches.
+
+### Runtime Configuration Portal
+
+When you power on a freshly flashed device, it creates a WiFi access point named OpenMQTTGateway or starting with OMG_. You can find detailed information about the [configuration portal here](portal.md).
+
+Connecting to this access point opens a portal where you configure your WiFi network, MQTT broker details, and optional security settings including TLS encryption and certificates. For devices with Ethernet, access the portal through the LAN IP address and configure WiFi as fallback connectivity.
+
+The portal accepts broker IP addresses or hostnames with mDNS support like homeassistant.local. Set a gateway password to protect future configuration changes, OTA updates, and web interface access.
+
+### Build-Time Configuration
+
+Alternatively, embed network and MQTT settings directly in firmware during the build process. Set parameters in `User_config.h` or add them as build flags in your PlatformIO environment definition. Store sensitive information in a separate `_env.ini` file excluded from version control.
+
+## Advanced Configuration Options
+
+Beyond basic connectivity, OpenMQTTGateway supports several [advanced features](advanced-configuration.md) that enhance security and integration capabilities.
+
+### Secure MQTT Connections
+
+For deployments over the internet or public networks, enable TLS encryption to secure communication between the gateway and MQTT broker. Configure your broker with a valid certificate and obtain the Certificate Authority certificate. The gateway can verify server identity against this certificate or connect with encryption without validation.
+
+Provide the CA certificate at build time in `default_server_cert.h` or paste it into the configuration portal. The gateway supports both self-signed certificates and those from public certificate authorities.
+
+### Home Assistant Auto-Discovery
+
+When you use [Home Assistant](https://www.home-assistant.io/) as your home automation platform, OpenMQTTGateway automatically creates device entries and sensors through Home Assistant's MQTT discovery protocol, enabled by default in all standard builds.
+
+Enable discovery in your Home Assistant MQTT integration settings and create a dedicated MQTT user. The gateway registers itself as a device and creates sensor entities automatically, appearing in Configuration → Devices section.
+
+### Topic Customization
+
+The gateway publishes messages to MQTT topics following the format `home/OpenMQTTGateway/GATEWAYtoMQTT`. Enable the `valueAsATopic` feature to append received values to the topic path, making topic-based filtering easier and avoiding warnings in certain controllers.
+
+## Next Steps
+
+After successfully uploading firmware and configuring your gateway, you can proceed to configure specific gateway modules for [RF](../setitup/rf.md), [IR](../setitup/ir.md), [Bluetooth](../setitup/ble.md), [LoRa](../setitup/lora.md), or other protocols you want to use. Each module has configuration options adjustable through MQTT commands or the web interface without rebuilding firmware.
+
+The [troubleshooting section](troubleshoot.md) covers common issues, but if you encounter problems not addressed here, the [OpenMQTTGateway community forum](https://community.openmqttgateway.com) provides an active place to ask questions and share solutions with other users.
diff --git a/docs/upload/board-selector.md b/docs/upload/board-selector.md
new file mode 100644
index 00000000..3c498cba
--- /dev/null
+++ b/docs/upload/board-selector.md
@@ -0,0 +1,10 @@
+---
+pageClass: table-generated-page
+---
+
+
+
+
+
\ No newline at end of file
diff --git a/docs/upload/web-install.md b/docs/upload/web-install.md
index aeb494f6..3547da57 100644
--- a/docs/upload/web-install.md
+++ b/docs/upload/web-install.md
@@ -2,7 +2,14 @@
pageClass: table-generated-page
---
-# (Option 1) Upload from the web
+## Select your firmware
+To upload firmware to your ESP device directly from this page, first connect your ESP device to a USB port on your computer. Then, choose the appropriate firmware from the available options. Next, click the **Connect** button and select the USB port where your ESP is plugged in. Wait for the upload process to finish. After the upload completes, you can set up your [WiFi and MQTT credentials](portal.md).
+
+
+
::: tip Running on a tablet or phone
If you want to use the BLE decoding capabilities of OpenMQTTGateway with a tablet or smartphone you can use [Theengs App](https://app.theengs.io/).
@@ -15,16 +22,8 @@ The correct driver to then select in the popup of this web install is
`/dev/cu.wchusbserialXXXXXXXXXXX`
:::
-You can upload the firmware to your ESP device directly from here.
-1. Plug in your ESP to a USB port.
-2. Select the firmware in the box below.
-3. Click the install button and choose the port that the ESP is connected to.
-4. Wait until the process is complete.
-5. Once completed you can configure your [WiFi and MQTT credentials](portal.md)
-
-Upload powered by [ESP Web Tools](https://esphome.github.io/esp-web-tools/)
## Using OpenMQTTGateway ?
Support open-source development through sponsorship and gain exclusive access to our private forum. Your questions, issues, and feature requests will receive priority attention, plus you'll gain insider access to our roadmap.
@@ -33,5 +32,4 @@ Support open-source development through sponsorship and gain exclusive access to
-## Environments characteristics
-The auto-generated table below describes the libraries and the modules of each board configuration.
+
diff --git a/docsgen/boards-info.js b/docsgen/boards-info.js
new file mode 100644
index 00000000..72717b94
--- /dev/null
+++ b/docsgen/boards-info.js
@@ -0,0 +1,232 @@
+'use strict';
+/**
+ * Universal parser for PlatformIO dependencies.
+ * Formats URLs and registry strings into a consistent "Registry Style":
+ * Name @ Version (provider:user)
+ */
+function smartFormat(dep) {
+ if (!dep) return "";
+ if (typeof dep !== 'string') return dep;
+
+ const cleanDep = dep.trim();
+
+ // Configuration for Git providers with specific regex for archives, releases, and git repos
+ const providers = [
+ {
+ id: 'gh',
+ name: 'github',
+ // Captures: 1. Author, 2. Repo, 3. Version from path (releases), 4. Version from filename/branch
+ regex: /github\.com\/([^/]+)\/([^/.]+)(?:\/(?:archive|releases\/download\/([^/]+)|tree)\/)?([^/]+)?(?:\.zip|\.git)?$/i
+ },
+ {
+ id: 'gl',
+ name: 'gitlab',
+ // Captures: 1. Author, 2. Repo, 3. Version from path, 4. Version from filename
+ regex: /gitlab\.com\/([^/]+)\/([^/.]+)(?:\/(?:-\/)?(?:archive|releases)\/([^/]+))?\/([^/]+)?(?:\.zip|\.git)?$/i
+ },
+ {
+ id: 'bb',
+ name: 'bitbucket',
+ // Captures: 1. Author, 2. Repo, 3. Version from path, 4. Version from filename
+ regex: /bitbucket\.org\/([^/]+)\/([^/.]+)(?:\/(?:get|downloads)\/([^/]+))?\/([^/]+)?(?:\.zip|\.git)?$/i
+ }
+ ];
+
+ // 1. Try to match against Git providers (GitHub, GitLab, Bitbucket)
+ for (const p of providers) {
+ const match = cleanDep.match(p.regex);
+ if (match) {
+ let [_, author, repo, pathVer, fileVer] = match;
+
+ // Prioritize version from path (typical in releases) over filename
+ let version = pathVer || fileVer || "latest";
+
+ // Clean up version string: remove extensions and 'v' prefix
+ version = version
+ .replace(/\.(zip|git|tar\.gz)$/i, '')
+ .replace(/^v(\d)/i, '$1'); // Removes 'v' only if followed by a number
+
+ // Avoid redundancy if the version string is identical to the repo name
+ if (version.toLowerCase() === repo.toLowerCase()) {
+ version = "latest";
+ }
+
+ return `${repo} @ ${version} (${p.id}:${author})`;
+ }
+ }
+
+ // 2. Fallback for Standard PlatformIO Registry format (e.g., owner/lib @ ^1.0.0)
+ if (cleanDep.includes('/') || cleanDep.includes('@')) {
+ const parts = cleanDep.split('@');
+ const fullName = parts[0].trim(); // Includes owner/name
+
+ // Clean up version if present
+ let version = "latest";
+ if (parts[1]) {
+ version = parts[1].trim().replace(/^[\^~=]/, '');
+ }
+
+ // Separate owner and library name for consistent formatting
+ if (fullName.includes('/')) {
+ const [owner, libName] = fullName.split('/');
+ return `${libName.trim()} @ ${version} (pio:${owner.trim()})`;
+ }
+
+ return `${fullName} @ ${version}`;
+ }
+
+ // 3. Return original string if no patterns match
+ return cleanDep;
+}
+
+function rowConfigFromPlatformIO() {
+ const { execSync } = require('child_process');
+
+ try {
+ const jsonConfig = execSync('pio project config --json-output').toString();
+ const config = JSON.parse(jsonConfig);
+ return config;
+ } catch (error) {
+ console.error("Make sure PlatformIO Core is installed and in PATH");
+ throw error;
+ }
+}
+
+function cleanValue(v) {
+ if (typeof v !== 'string') return v;
+ return v
+ .replace(/{/g, '')
+ .replace(/}/g, '')
+ .replace(/\$/g, '')
+ .replace(/env:/g, '')
+ .replace(/'/g, '')
+ .replace(/-D/g, '');
+}
+
+function convertJsonToSections(jsonConfig) {
+ const sections = {};
+ jsonConfig.forEach(([sectionName, configArray]) => {
+ sections[sectionName] = {};
+ configArray.forEach(([key, value]) => {
+ sections[sectionName][key] = value;
+ });
+ });
+ return sections;
+}
+
+function cleanLibraries(raw) {
+ if (!raw) return [];
+ if (typeof raw === 'string') {
+ raw = raw.split(',')
+ }
+ return raw.map((dep) => smartFormat(dep));
+}
+
+function extractModulesFromFlags(flags) {
+ if (!flags) return [];
+ let flagArray = [];
+ if (Array.isArray(flags)) {
+ flagArray = flags;
+ } else if (typeof flags === 'string') {
+ flagArray = flags.split(',').map(s => s.trim()).filter(s => s.length > 0);
+ } else {
+ return [];
+ }
+ const modules = [];
+ flagArray.forEach((flag) => {
+ // Match -DZmoduleName, allowing surrounding quotes
+ const match = flag.match(/^['" ]*-DZ([^=]+)/);
+ if (match) {
+ const moduleName = match[1];
+ // Additional constraint: must contain 'gateway', 'sensor', or 'actuator'
+ if (moduleName.includes('gateway') || moduleName.includes('sensor') || moduleName.includes('actuator')) {
+ modules.push(moduleName);
+ }
+ }
+ //if MQTT_BROKER_MODE = true then modules.push("MQTT Broker Mode");
+ const brokerMatch = flag.match(/^['" ]*-DMQTT_BROKER_MODE(?:=([^'"\s]+))?/);
+ if (brokerMatch) {
+ const value = brokerMatch[1];
+ // Add only if not explicitly set to false (case insensitive)
+ if (!value || value.toLowerCase() !== 'false') {
+ modules.push("MQTT Broker Mode");
+ }
+ }
+
+ });
+ return modules;
+}
+
+function collectBoardsInformations(sections, { includeTests = false } = {}) {
+ const rows = [];
+
+ Object.entries(sections).forEach(([section, items]) => {
+ if (!section.includes('env:')) return;
+ if (!includeTests && section.includes('-test')) return;
+
+ const env = section.replace('env:', '');
+ let uc = '';
+ let hardware = '';
+ let description = '';
+ let modules = [];
+ let platform = '';
+ let partitions = '';
+ let libraries = [];
+ let options = [];
+ let customImg = '';
+
+ Object.entries(items).forEach(([k, raw]) => {
+ const v = cleanValue(raw);
+
+
+ if (k === 'board') uc = v;
+ if (k === 'platform') platform = smartFormat(v);
+ if (k === 'board_build.partitions') partitions = v;
+ if (k === 'custom_description') description = v;
+ if (k === 'custom_hardware') hardware = v;
+ if (k === 'custom_img') customImg = v;
+
+ if (k === 'lib_deps') {
+ libraries = cleanLibraries(raw);
+ }
+
+ if (k === 'build_flags') {
+ options = v;
+ modules = extractModulesFromFlags(v);
+ }
+ });
+
+ rows.push({
+ Environment: env,
+ uC: uc,
+ Hardware: hardware,
+ Description: description,
+ Modules: modules,
+ Platform: platform,
+ Partitions: partitions,
+ Libraries: libraries,
+ Options: options,
+ CustomImg: customImg
+ });
+ });
+
+ rows.sort((a, b) => a.Environment.localeCompare(b.Environment, 'en', { sensitivity: 'base' }));
+ return rows;
+}
+
+function loadBoardsInfo(options = {}) {
+ const { includeTests = false } = options;
+ const config = rowConfigFromPlatformIO();
+ const sections = convertJsonToSections(config);
+ return collectBoardsInformations(sections, { includeTests });
+}
+
+function ensureDir(dir) {
+ const fs = require('fs');
+ if (!fs.existsSync(dir)) fs.mkdirSync(dir, { recursive: true });
+}
+
+module.exports = {
+ loadBoardsInfo,
+ ensureDir
+};
diff --git a/docsgen/common_wu.js b/docsgen/common_wu.js
new file mode 100644
index 00000000..bccc3078
--- /dev/null
+++ b/docsgen/common_wu.js
@@ -0,0 +1,77 @@
+// Common templates and constants for web installer manifest generation
+// Used by: scripts/gen_wu.js
+
+const mf_temp32 = (vars) => `{
+ "name": "OpenMQTTGateway",
+ "new_install_prompt_erase": true,
+ "builds": [
+ {
+ "chipFamily": "ESP32",
+ "improv": false,
+ "parts": [
+ { "path": "${vars.cp}${vars.bl}", "offset": 4096 },
+ { "path": "${vars.cp}${vars.part}", "offset": 32768 },
+ { "path": "${vars.cp}${vars.boot}", "offset": 57344 },
+ { "path": "${vars.cp}${vars.bin}", "offset": 65536 }
+ ]
+ }
+ ]
+}`;
+
+const mf_temp32c3 = (vars) => `{
+ "name": "OpenMQTTGateway",
+ "new_install_prompt_erase": true,
+ "builds": [
+ {
+ "chipFamily": "ESP32-C3",
+ "improv": false,
+ "parts": [
+ { "path": "${vars.cp}${vars.bl}", "offset": 0 },
+ { "path": "${vars.cp}${vars.part}", "offset": 32768 },
+ { "path": "${vars.cp}${vars.boot}", "offset": 57344 },
+ { "path": "${vars.cp}${vars.bin}", "offset": 65536 }
+ ]
+ }
+ ]
+}`;
+
+const mf_temp32s3 = (vars) => `{
+ "name": "OpenMQTTGateway",
+ "new_install_prompt_erase": true,
+ "builds": [
+ {
+ "chipFamily": "ESP32-S3",
+ "improv": false,
+ "parts": [
+ { "path": "${vars.cp}${vars.bl}", "offset": 0 },
+ { "path": "${vars.cp}${vars.part}", "offset": 32768 },
+ { "path": "${vars.cp}${vars.boot}", "offset": 57344 },
+ { "path": "${vars.cp}${vars.bin}", "offset": 65536 }
+ ]
+ }
+ ]
+}`;
+
+const mf_temp8266 = (vars) => `{
+ "name": "OpenMQTTGateway",
+ "new_install_prompt_erase": true,
+ "builds": [
+ {
+ "chipFamily": "ESP8266",
+ "parts": [{ "path": "${vars.cp}${vars.bin}", "offset": 0 }]
+ }
+ ]
+}`;
+
+
+const cors_proxy = ''; // 'https://cors.bridged.cc/'
+const esp32_boot = 'https://github.com/espressif/arduino-esp32/raw/2.0.7/tools/partitions/boot_app0.bin';
+
+module.exports = {
+ mf_temp32,
+ mf_temp32c3,
+ mf_temp32s3,
+ mf_temp8266,
+ cors_proxy,
+ esp32_boot
+};
diff --git a/docsgen/gen_wu.js b/docsgen/gen_wu.js
new file mode 100644
index 00000000..5c5ed630
--- /dev/null
+++ b/docsgen/gen_wu.js
@@ -0,0 +1,349 @@
+#!/usr/bin/env node
+
+// Creates web installer manifests for ESP Web Tools firmware installation
+
+const fs = require('fs');
+const path = require('path');
+const https = require('https');
+const { loadBoardsInfo, ensureDir } = require('./boards-info');
+
+const {
+ mf_temp32,
+ mf_temp32c3,
+ mf_temp32s3,
+ mf_temp8266,
+ cors_proxy,
+ esp32_boot
+} = require('./common_wu.js');
+
+// ============================================================================
+// Directory Configuration
+// ============================================================================
+
+// Base directories
+const ROOT_DIR = path.join(__dirname, '..');
+const DOCS_DIR = path.join(ROOT_DIR, 'docs');
+const VUEPRESS_DIR = path.join(DOCS_DIR, '.vuepress');
+const PUBLIC_DIR = path.join(VUEPRESS_DIR, 'public');
+const COMPONENTS_DIR = path.join(VUEPRESS_DIR, 'components');
+const ARTIFACTS_DIR = path.join(ROOT_DIR, 'generated', 'artifacts');
+
+// Feature-specific directories
+const FIRMWARE_SRC_DIR = path.join(ARTIFACTS_DIR, 'firmware_build');
+const FIRMWARE_BUILD_DIR = path.join(PUBLIC_DIR, 'firmware_build');
+const BOARDS_INFO_FILE = path.join(PUBLIC_DIR, 'boards-info.json');
+
+// Configuration files
+const DEFAULTS_CONFIG_PATH = path.join(VUEPRESS_DIR, 'defaults.json');
+const META_CONFIG_PATH = path.join(VUEPRESS_DIR, 'meta.json');
+
+
+
+let meta = require(DEFAULTS_CONFIG_PATH);
+try {
+ const meta_overload = require(META_CONFIG_PATH);
+ meta = { ...meta, ...meta_overload };
+} catch (e) {
+ console.warn('meta.json not found or not valid. Using default configuration.');
+}
+
+// Parse command line arguments
+const args = process.argv.slice(2);
+const dev = args.includes('--dev') || meta.mode === 'dev';
+const repo = meta.repo || '1technophile/OpenMQTTGateway';
+const firmwareManifestFolder = dev ? `/dev/firmware_build/` : `/firmware_build/`;
+
+// ============================================================================
+// Utility Functions
+// ============================================================================
+
+function ensureFirmwareArtifacts() {
+ if (!fs.existsSync(FIRMWARE_SRC_DIR)) {
+ throw new Error(`Missing firmware artifacts in ${FIRMWARE_SRC_DIR}. Run "ci.sh build ..." first to populate this folder.`);
+ }
+ console.log(`Found firmware artifacts in: ${FIRMWARE_SRC_DIR}`);
+}
+// Replace version_tag in template and write to destination
+function renderVersionTemplate(templatePath, outputPath, version) {
+ if (!fs.existsSync(templatePath)) {
+ throw new Error(`Template not found: ${templatePath}`);
+ }
+ const content = fs.readFileSync(templatePath, 'utf8').replace(/version_tag/g, version);
+ fs.writeFileSync(outputPath, content);
+ console.log(`Generated version file from template: ${outputPath}`);
+}
+
+/**
+ * Download file from URL
+ */
+function downloadFile(url) {
+ return new Promise((resolve, reject) => {
+ https.get(url, (response) => {
+ // Handle redirects
+ if (response.statusCode === 302 || response.statusCode === 301) {
+ return downloadFile(response.headers.location).then(resolve).catch(reject);
+ }
+ if (response.statusCode !== 200) {
+ return reject(new Error(`Failed to download: ${response.statusCode}`));
+ }
+ const chunks = [];
+ response.on('data', (chunk) => chunks.push(chunk));
+ response.on('end', () => resolve(Buffer.concat(chunks)));
+ response.on('error', reject);
+ }).on('error', reject);
+ });
+}
+
+/**
+ * Fetch JSON from URL
+ */
+function fetchJson(url) {
+ return new Promise((resolve, reject) => {
+ https.get(url, { headers: { 'User-Agent': 'OpenMQTTGateway-Script' } }, (response) => {
+ if (response.statusCode !== 200) {
+ return reject(new Error(`Failed to fetch: ${response.statusCode}`));
+ }
+ let data = '';
+ response.on('data', (chunk) => data += chunk);
+ response.on('end', () => {
+ try {
+ resolve(JSON.parse(data));
+ } catch (e) {
+ reject(e);
+ }
+ });
+ response.on('error', reject);
+ }).on('error', reject);
+ });
+}
+
+/**
+ * Download and save asset
+ */
+async function downloadAsset(asset, destPath) {
+ const buffer = await downloadFile(asset.browser_download_url);
+ const filename = asset.browser_download_url.split('/').pop();
+ fs.writeFileSync(path.join(destPath, filename), buffer);
+ console.log(`Downloaded asset: ${filename} to ${destPath}`);
+}
+
+/**
+ * Create manifest and Vue option for a firmware
+ * Partition path uses filename only (matches Python; split is redundant but harmless)
+ */
+function createManifest(name, templateFn) {
+ const fw = name.split('-firmware')[0];
+ const man_file = fw + '.manifest.json';
+ const fwp_name = fw + '-partitions.bin';
+ const fwb_name = fw + '-bootloader.bin';
+
+ // Use filename to mirror Python behavior (no directories present today)
+ const partPath = fwp_name.split('/').pop();
+
+ const mani_str = templateFn({
+ cp: cors_proxy,
+ part: firmwareManifestFolder + partPath,
+ bin: firmwareManifestFolder + name,
+ bl: firmwareManifestFolder + fwb_name,
+ boot: firmwareManifestFolder + esp32_boot.split('/').pop()
+ });
+
+ const outPath = path.join(FIRMWARE_BUILD_DIR, man_file);
+ fs.writeFileSync(outPath, mani_str);
+ console.log(`Created manifest for ${fw}: ${outPath}`);
+}
+
+/**
+ * Create manifest for ESP8266
+ * Python adds manif_folder when writing to file, not in return
+ */
+function createManifest8266(name) {
+ const fw = name.split('-firmware')[0];
+ const man_file = fw + '.manifest.json';
+
+ const mani_str = mf_temp8266({
+ cp: cors_proxy,
+ bin: firmwareManifestFolder + name
+ });
+
+ const outPath = path.join(FIRMWARE_BUILD_DIR, man_file);
+ fs.writeFileSync(outPath, mani_str);
+ console.log(`Created manifest for ${fw} (ESP8266): ${outPath}`);
+
+}
+
+/**
+ * Device type matchers
+ */
+const ESP32_NAMES = ['esp32', 'ttgo', 'heltec', 'thingpulse', 'theengs', 'lilygo', 'shelly', 'tinypico'];
+const ESP8266_NAMES = ['nodemcu', 'sonoff', 'rf-wifi-gateway', 'manual-wifi-test', 'rfbridge'];
+
+const deviceMatchers = {
+ esp32: (name) => name.includes('firmware.bin') &&
+ !name.includes('esp32c3') && !name.includes('esp32s3') &&
+ ESP32_NAMES.some(key => name.includes(key)),
+
+ esp32c3: (name) => name.includes('firmware.bin') && name.includes('esp32c3'),
+
+ esp32s3: (name) => name.includes('firmware.bin') && name.includes('esp32s3'),
+
+ esp8266: (name) => name.includes('firmware.bin') &&
+ ESP8266_NAMES.some(key => name.includes(key))
+};
+
+/**
+ * Setup dev environment
+ */
+async function setupDevEnvironment() {
+ console.log('DEV mode: preparing web upload files...');
+ ensureFirmwareArtifacts();
+ // Generate OTA latest version definition from template
+ const tpl = path.join(__dirname, 'latest_version_dev.json.tpl');
+ renderVersionTemplate(tpl, path.join(FIRMWARE_BUILD_DIR, 'latest_version_dev.json'), meta.version);
+
+ // Copy the binaries from FIRMWARE_SRC_DIR to FIRMWARE_BUILD_DIR
+ const files = fs.readdirSync(FIRMWARE_SRC_DIR);
+ let copied = 0;
+ for (const name of files) {
+ if (name.includes('.bin')) {
+ fs.copyFileSync(
+ path.join(FIRMWARE_SRC_DIR, name),
+ path.join(FIRMWARE_BUILD_DIR, name)
+ );
+ copied++;
+ console.log(`Copied binary: ${name}`);
+ }
+ }
+ console.log(`Copied ${copied} firmware binaries to ${FIRMWARE_BUILD_DIR}`);
+}
+
+/**
+ * Setup release environment
+ */
+async function setupReleaseEnvironment() {
+ console.log('RELEASE mode: downloading and preparing web upload files...');
+
+ // Generate OTA latest version definition from template
+ const tpl = path.join(__dirname, 'latest_version.json.tpl');
+ renderVersionTemplate(tpl, path.join(FIRMWARE_BUILD_DIR, 'latest_version.json'), meta.version);
+
+ const releaseUrl = `https://api.github.com/repos/${repo}/releases/latest`;
+ console.log(`Fetching latest release info from: ${releaseUrl}`);
+ const rel_data = await fetchJson(releaseUrl);
+
+ if (!rel_data.assets) {
+ console.error('No assets found in the latest release!');
+ process.exit(1);
+ }
+
+ // Download all assets
+ let downloaded = 0;
+ for (const asset of rel_data.assets) {
+ const name = asset.name;
+ if (name.includes('firmware.bin') ||
+ name.includes('partitions.bin') ||
+ name.includes('bootloader.bin')) {
+ await downloadAsset(asset, FIRMWARE_BUILD_DIR);
+ downloaded++;
+ }
+ }
+ console.log(`Downloaded ${downloaded} firmware assets to ${FIRMWARE_BUILD_DIR}`);
+}
+
+/**
+ * Process firmware files and generate manifests
+ */
+function processFirmwareFiles(files) {
+ let manifestCount = 0;
+ for (const name of files) {
+ if (deviceMatchers.esp32(name)) {
+ createManifest(name, mf_temp32);
+ manifestCount++;
+ }
+ if (deviceMatchers.esp32c3(name)) {
+ createManifest(name, mf_temp32c3);
+ manifestCount++;
+ }
+ if (deviceMatchers.esp32s3(name)) {
+ createManifest(name, mf_temp32s3);
+ manifestCount++;
+ }
+ if (deviceMatchers.esp8266(name)) {
+ createManifest8266(name);
+ manifestCount++;
+ }
+ }
+ console.log(`Generated ${manifestCount} manifest files in ${FIRMWARE_BUILD_DIR}`);
+
+}
+
+/**
+ * Main execution function
+ */
+// ===================== OpenMQTTGateway Web Uploader Manifest Generator =====================
+// ===================== MAIN SCRIPT STARTS HERE =====================
+async function main() {
+ console.log('================================================================================');
+ console.log(' OpenMQTTGateway Web Uploader Manifest Generator - START');
+ console.log('================================================================================');
+
+ // === [1] Load and generate boards info ===
+ console.log('\n[1/4] Generating boards-info.json ...');
+ const boardsInfo = loadBoardsInfo({ verbose: 0 });
+ const boardsJson = boardsInfo.map((row) => ({
+ environment: row.Environment,
+ hardware: row.Hardware,
+ description: row.Description,
+ microcontroller: row.uC,
+ modules: row.Modules.filter(Boolean),
+ platform: row.Platform,
+ partitions: row.Partitions,
+ libraries: row.Libraries.filter(Boolean),
+ options: row.Options,
+ customImg: row.CustomImg
+ }));
+ ensureDir(path.dirname(BOARDS_INFO_FILE));
+ fs.writeFileSync(BOARDS_INFO_FILE, JSON.stringify(boardsJson, null, 2), 'utf8');
+ console.log(`Generated boards-info.json with ${boardsJson.length} boards: ${BOARDS_INFO_FILE}`);
+
+ // === [2] Ensure output directory ===
+ console.log('\n[2/4] Ensuring output directory ...');
+ ensureDir(FIRMWARE_BUILD_DIR);
+ console.log(`Ensured output directory exists: ${FIRMWARE_BUILD_DIR}`);
+
+ // === [3] Setup environment (dev or release) ===
+ console.log('\n[3/4] Preparing firmware files ...');
+ try {
+ if (dev) {
+ await setupDevEnvironment();
+ } else {
+ await setupReleaseEnvironment();
+ }
+ } catch (error) {
+ console.error(`Error setting up environment: ${error.message}`);
+ process.exit(1);
+ }
+
+ // === [4] Download boot binary and generate manifests ===
+ console.log('\n[4/4] Downloading boot binary and generating manifests ...');
+ console.log(`Downloading boot binary: ${esp32_boot}`);
+ const boot_bin = await downloadFile(esp32_boot);
+ const boot_filename = esp32_boot.split('/').pop();
+ fs.writeFileSync(path.join(FIRMWARE_BUILD_DIR, boot_filename), boot_bin);
+ console.log(`Saved boot binary as: ${boot_filename}`);
+
+ const files = fs.readdirSync(FIRMWARE_BUILD_DIR).sort();
+ console.log(`Processing firmware files in ${FIRMWARE_BUILD_DIR}...`);
+ processFirmwareFiles(files);
+
+ console.log('\n================================================================================');
+ console.log(' OpenMQTTGateway Web Uploader Manifest Generator - END');
+ console.log('================================================================================');
+}
+// ===================== MAIN SCRIPT ENDS HERE =====================
+
+// Run main function
+main().catch(error => {
+ console.error('Fatal error:', error);
+ process.exit(1);
+});
diff --git a/scripts/latest_version.json b/docsgen/latest_version.json.tpl
similarity index 100%
rename from scripts/latest_version.json
rename to docsgen/latest_version.json.tpl
diff --git a/scripts/latest_version_dev.json b/docsgen/latest_version_dev.json.tpl
similarity index 100%
rename from scripts/latest_version_dev.json
rename to docsgen/latest_version_dev.json.tpl
diff --git a/environments.ini b/environments.ini
index a849a343..d464ff6c 100644
--- a/environments.ini
+++ b/environments.ini
@@ -46,7 +46,7 @@ build_flags =
board_build.flash_mode = dout
board_build.ldscript = eagle.flash.1m64.ld ;this frees more space for firmware uplad via OTA.
;extra_scripts = scripts/compressFirmware.py ;uncomment this to compress the firmware. This helps updating e.g. Sonoff RF Bridge via OTA flash by saving space for the uploaded firmware.
-custom_description = RF gateway for the Sonoff RF Bridge requiring direct hack, relying on ESPilight library, [tutorial](https://1technophile.blogspot.com/2019/04/sonoff-rf-bridge-pilight-or-how-to.html).
+custom_description = 'RF gateway for the Sonoff RF Bridge requiring direct hack, relying on ESPilight library,