Commit Graph

758 Commits

Author SHA1 Message Date
Alessandro Staniscia
98481c5145 [SITE] Renew the web board presentation and the ESP32 web upload + [SYS] Security checks (#2277)
* Refactor GitHub Actions workflows for build, documentation, and linting

- Consolidated build logic into reusable workflows (`task-build.yml` and `task-docs.yml`) to reduce duplication across multiple workflows.
- Introduced `environments.json` to centralize the list of PlatformIO build environments, improving maintainability and clarity.
- Updated `build.yml` and `build_and_docs_to_dev.yml` to utilize the new reusable workflows and environment definitions.
- Enhanced `release.yml` to streamline the release process and integrate documentation generation.
- Created reusable linting workflow (`task-lint.yml`) to standardize code formatting checks across the repository.
- Simplified manual documentation workflow by leveraging the new reusable documentation workflow.
- Improved artifact management and retention policies across workflows.
- Updated dependencies and versions in workflows to ensure compatibility and performance.

CI/CD pipeline agnostic of Workflow Engine and integrated on github actions

- Implemented ci.sh for orchestrating the complete build pipeline.
- Created ci_00_config.sh for centralized configuration of build scripts.
- Created ci_build_firmware.sh for building firmware for specified PlatformIO environments.
- Created ci_prepare_artifacts.sh for preparing firmware artifacts for upload or deployment.
- Created ci_set_version.sh for updating version tags in firmware configuration files.
- Created ci_build.sh to orchestrate the complete build pipeline.
- Created ci_qa.sh for code linting and formatting checks using clang-format.
- Created ci_site.sh for building and deploying VuePress documentation with version management.
- Implemented checks for required tools and dependencies in the new scripts.
- Improved internal scripts for better error handling and logging.

UPDATE the web installer manifest generation and update documentation structure
- Enhanced ci_list-env.sh to list environments from a JSON file.
- Replaced  common_wu.py and gen_wu.py scripts with new npm scripts for site generation and previewing on docsgen/gen_wu.js
- Replaced  generate_board_docs.py with docsgen/generated_board_docs.js
- Added new npm scripts for integration of site generation on build phase.
- Created preview_site.js to serve locally generated site over HTTPS with improved error handling.
- Added new CI environments for CI builds in environments.json.
- Deleted lint.yml as part of workflow cleanup.
- Enhanced task-build.yml to include linting as a job and added support for specifying PlatformIO version.
- Improved task-docs.yml to handle versioning more effectively and added clean option.

Enhance documentation
- ADD CLEAR Mark of development version of site
- Updated README.md to include detailed workflow dependencies and relationships using mermaid diagrams.
- Improved development.md with a quick checklist for contributors and clarified the code style guide.
- Enhanced quick_start.md with tips for contributors and streamlined the workflow explanation.

LINT FIX
- Refined User_config.h for better formatting consistency.
- Adjusted blufi.cpp and gatewayBT.cpp for improved code readability and consistency in formatting.
- Updated gatewaySERIAL.cpp and mqttDiscovery.cpp to enhance logging error messages.
- Improved sensorDS1820.cpp for better logging of device information.

Add security scan workflows for vulnerability detection

Add SBOM generation and upload to release workflow; update security scan summary handling

Add shellcheck suppor + FIX shellcheck warning

Enhance documentation for CI/CD scripts and workflows, adding details for security scanning and SBOM generation processes

Fix formatting and alignment in BLE connection handling

Reviewed the full web board presentation and the ESP32 web upload. The project uses a modern pattern where data is divided from the presentation layer.

- Removed the `generate_board_docs` script.
- Updated the `gen_wu` script in order to generate `boards-info.json`: the fail that containe all information about the configuration
- Created and isolate the file `boards-info.js` to streamline the parsing of PlatformIO dependencies, modules, environments and improve the handling of library information.
- Introduced vuepress component `BoardEnvironmentTable.vue` that render `boards-info.json` as UI card component
- Introduced vuepress component `FlashEnvironmentSelector.vue` that render a selectred environment from  `boards-info.json` and provide esp-web-upload feature on it
- Introduced a new board page `board-selector.md` for improved firmware selection.
- Updated `web-install.md` to enhance the firmware upload process, including a new board environment table.
- Enhanced custom descriptions in `environments.ini` to include HTML links for better user guidance and board image link

Add CC1101 initialization improvements and logging enhancements
Add installation step for PlatformIO dependencies in documentation workflow

Remove ci_set_version.sh script and associated versioning functionality

* Fix comment provisined

Fix PlatformIO version input reference in documentation workflow

Remove outdated Squeezelite-ESP32 installer documentation
2026-03-09 07:47:30 -05:00
Ben Fulton
df1204d92f [DISC] Add HASS_UNIT_PPM to availableHASSUnits array (#2279)
HASS_UNIT_PPM is defined in config_mqttDiscovery.h but was missing from
the availableHASSUnits validation array in mqttDiscovery.cpp. This caused
unit_of_measurement to be silently dropped for CO2 sensors (like the
Govee H5140) during MQTT discovery, resulting in Home Assistant warnings
about invalid units.
2026-02-15 15:35:22 -06:00
Florian
fd433c220e [BLE] Add BM6 Battery Monitor connection support (#2274)
* [BLE] Add BM6 Battery Monitor connection support

Implement BLE connection support for BM6 Battery Monitor devices,
following the BM2 pattern with BM6-specific protocol handling.

Key features:
- Encrypted command/response using AES-128 CBC
- Read voltage, temperature, and State of Charge (SoC)
- Home Assistant MQTT Discovery support
- Device tracker integration

Technical implementation:
- Add BM6_connect class with encrypted command handling
- Encryption key (static const): "leagend\xff\xfe0100009"
- Parse hex string positions for voltage (15-17), temp (8-9), SoC (12-13)
- Add discovery with 3 sensors (voltage, temperature, battery %)
- Validate message signature (D15507)
- Add canWrite() check before writing to characteristic
- Empty stub for non-ESP32 builds

Code quality improvements:
- Extract AES key as static constant to avoid duplication
- Use shared jsonVoltBM template for both BM2 and BM6
- Clean up commented-out reasoning in parsing logic

Based on reverse engineering from:
https://github.com/JeffWDH/bm6-battery-monitor

Files modified:
- main/gatewayBLEConnect.h: Add BM6_connect class
- main/gatewayBLEConnect.cpp: Implement BM6 connection logic
- main/gatewayBT.cpp: Add detection, connection, and discovery handlers
- main/config_mqttDiscovery.h: Rename jsonVoltBM2 to jsonVoltBM

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* Add BM6 to exceptions

---------

Co-authored-by: Florian <1technophile@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-24 17:16:03 -06:00
Florian
f52bc6c11e [SYS] Fix: Set WiFi hostname to gateway name (#2272)
* Fix: Set WiFi hostname to gateway name

Add WiFi.setHostname() calls to ensure the ESP's WiFi hostname matches
the configured gateway name instead of using the default ESP hostname.

- Add wifiManager.setHostname(gateway_name) in setupWiFiManager() for
  WiFiManager-based setups
- Add WiFi.setHostname(gateway_name) in setupWiFiFromBuild() for manual
  WiFi setup configurations (ESPWifiManualSetup)

This matches the existing behavior for Ethernet connections where
ETH.setHostname(gateway_name) is already called.

Fixes #2150

Co-authored-by: Florian <1technophile@users.noreply.github.com>

* Change WiFi mode position

* Fix setHostname when using WiFi Manager

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Florian <1technophile@users.noreply.github.com>
2026-01-21 08:22:00 -06:00
Florian
32664499f3 [LORA] Fix syncword not persisting after reset (#2271)
The bug was a race condition where saved configuration was applied
to the LoRa hardware BEFORE LoRa.begin() was called, causing settings
like syncword to be lost after reset/power cycle.

Changes:
- Created LORAConfig_apply() to centralize hardware configuration
- Removed hardware application from LORAConfig_fromJson() (now only
  updates struct values)
- Call LORAConfig_apply() after LoRa.begin() in setupLORA()
- Call LORAConfig_apply() after LORAConfig_fromJson() in runtime
  handlers (MQTT/WebUI) since hardware is already initialized
- Fixed syncword logging to properly show 'unchanged' vs 'changed'

Fixes #2270

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Florian <1technophile@users.noreply.github.com>
2026-01-10 11:59:41 -06:00
Florian
e01b22695a [RF] Fix CC1101 initialization on boot with saved RF config (#2269)
When a saved RF configuration exists in NVS, loadFromStorage() was loading
the config but not calling iRFReceiver.enable(), which meant initCC1101()
was never invoked during boot. This caused the CC1101 to not receive signals
until the user manually toggled the RF receiver via the WebUI.

This fix adds a reinitReceiver parameter to loadFromStorage() that controls
whether to disable/enable the receiver after loading config. By default it's
true (for boot-time initialization), but loadFromMessage() passes false to
avoid double initialization when loading config from MQTT messages.

Fixes #2264

Co-authored-by: Florian <1technophile@users.noreply.github.com>
Co-authored-by: Odyno <odyno@users.noreply.github.com>

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2026-01-06 05:47:49 -06:00
Alessandro Staniscia
c6b2aae965 Refactor RF Configuration Management (#2245)
- Introduced RFConfiguration class to encapsulate RF settings and operations.
- Replaced direct usage of RFConfig structure with iRFConfig instance across multiple files.
- Updated frequency handling in actuatorSomfy, gatewayPilight, gatewayRF, and gatewayRF2 to use iRFConfig.
- Modified webUI to interact with iRFConfig for RF settings management.
- Removed deprecated RFConfig structure and related functions.
- Enhanced JSON handling for RF configuration loading and saving.
- Improved logging for RF configuration operations.
2025-12-07 09:56:15 -06:00
Ryan Powell
75f9d30919 [SYS] Update to Arduino 3.3.3 and use custom libs (#2248)
* Update Arduino core to v3.3.2 and use custom Arduino libs

* Update Python version and install uv in build.yml (#2247)

Updated Python version and added uv installation step.

---------

Co-authored-by: Ryan Powell <ryan@nable-embedded.io>

* fix builds

* Update scripts/add_c_flags.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fetch archives instead of cloning libraries

* Update to Arduino 3.3.3

---------

Co-authored-by: h2zero <powellperalata@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-10 07:36:09 -06:00
Northern Man
382fa5e9f2 [SYS] Fix for build error with MDNS_SD (#2239)
* Fix for build error with MDNS_SD

* Update environments.ini
2025-09-16 05:38:43 -05:00
Florian
32174e37a5 [SYS] Reduce Logs flash footprint (#2232)
By building only the log level required into LOG_LEVEL

Co-authored-by: Florian <1technophile@users.noreply.github.com>
2025-08-19 09:15:41 -05:00
Florian
6e6b54b05e [DISC] Ram usage and flash size optimization (#2223)
Fix duplicate W and keep units and classes in flash
Use macro for discovery classes and units
[BREAKING] change gateway sensor unique id
module - parameter
Standardize the use of createDiscoveryFromList

Co-authored-by: Florian <1technophile@users.noreply.github.com>
2025-08-17 06:27:42 -05:00
DigiH
51bdc2d080 [BT] Victron decryption adjustments (#2228) 2025-08-16 15:43:54 +02:00
DigiH
14ceba5bee [BT] Consistent checking for BLEDecryptor as a Boolean (#2230)
Consistent checking for BLEDecryptor as a Boolean, not just as a Define, so that
…
…

also behaves correctly when set to false.
2025-08-16 15:03:19 +02:00
DigiH
5add388fbc [BT] Typo fix to actually have Victron devices decrypted (#2226) 2025-08-13 14:34:00 +02:00
Peter Lambrechtsen
75b3a3657e Add RSSI as diagnostic entity_category (#2224) 2025-08-12 07:11:42 -05:00
Peter Lambrechtsen
6d1b7935c4 [DISC] Replace Home Assistant Auto Discovery MQTT keys with shortened abbreviations (#2221) 2025-08-10 10:34:17 -05:00
Peter Lambrechtsen
ddb9cc67cc [DISC] Fix cosmetic issue in Home Assistant where devices don't link to gateway (#2220)
* Fix cosmetic issue in HA where devices don't link to gateway

* Update via_device to use getMacAddress() to get the OMG device mac address
2025-08-07 06:52:38 -05:00
Peter Lambrechtsen
2d29ee4524 [BLE] Add support for decrypting PVVX, BTHome v2 and Victron BLE frames (#2219)
* Changing WebUI to include display device name, and change it to select drop down rather than checkbox

* Fix mqttDiscovery to require WebUI and ESP32 for displayDeviceName

* Fix mqttDiscovery to require WebUI and ESP32 and ESP8266 for displayDeviceName and ForceDeviceName

* Changing WebUI to include display device name, and change it to select drop down rather than checkbox

* Fixes for WebUI and BT for supporting custom setting Display name

* Fixes for WebUI and BT for supporting custom setting Display name

* Move DISPLAY_DEVICE_NAME to User_config.h

* Update docs to include change for Display temperature

* Update docs to include change for Display temperature

* Fix minor cosmetic bug where devices were not linking in HA to the gateway using via_device as it should be the gateway mac address not name

* Add support for decrypting BTHome v2 frames

* Add support for decrypting BTHome v2 frames

* Add support for decrypting BTHome v2 frames

* BTHome fix issue with theengs-plug

* BTHome fix issue with theengs-plug

* Adding support for all BLE encrypted methods, support in UI and gatewayBT for specific MACAddress AES Keys

* Fix lint

* Fix build issue with theengs-bridge-v11 and esp32dev-all-test and revert the documentation to Units of measurement displayed

* Revert docs

* Revert displayDeviceName and Units of measurement

* Revert displayDeviceName and Units of measurement

* Revert displayDeviceName and Units of measurement

* Revert minor typo

* Revert minor typo

* Revert minor typo

* Bug in Victron as nonce should be 16 bytes

* Shortened the client side javascript for BLE key validation that is commented out due to image constrains on theengs-bridge-v11
2025-08-06 19:59:33 -05:00
Ryan Powell
a0d5b7d181 [BLE] Fix crash if BLE stack is not initialized when setting advertising. (#2212) 2025-08-05 07:08:03 -05:00
Ryan Powell
27a6ec592b [BLE] Add status to Blufi advertising. (#2198)
* blufi status

* Make the manufacturer ID configurable at build time.

* fix missing break.

* Update blufi gateway state when changed

---------

Co-authored-by: h2zero <powellperalata@gmail.com>
2025-07-07 22:54:52 +02:00
peshovec
162a4d3ae9 [RF] RF2, Pilight : fixups to RF2 transmit when using CC1100; fixups to Pilight when raw enabled (#2207) 2025-06-28 14:51:15 -05:00
DigiH
e47c145bad [RF] Fix discovery for negative Ids (#2194) (#2196) 2025-05-23 19:27:34 -05:00
DigiH
828f0a226e [DISC] device class added and Units sorted (#2189)
device class added and Units sorted alphabetically for easier lookup and adding missing units
2025-05-18 20:35:44 +02:00
Ryan Powell
301b3eecd8 [SYS] Misc bug fixes with strict compilation (#2188) 2025-05-17 17:36:07 -05:00
DigiH
6175e03bc8 [DISC] Additional device classes (#2185) 2025-05-17 01:42:40 +02:00
DigiH
f0491e188c [BT] MQTT corruption fix (#2183)
Storing a copy of the ID instead of a pointer to a local variable to avoid MQTT publishing id topic and id property corruption.
2025-05-16 22:15:57 +02:00
Ryan Powell
585df9a420 [SYS] Update arduino core to 3.1.1 + refactor ino to cpp (#2177)
* Update arduino core to 3.1.1

* Fix Blufi build

* Update arduinojson, fix build errors with idf

* Fix narrowing

* fix RF builds

* WIP-Convert ino files to cpp

* Fix pilight build

* Fix Somfy actuator build.

* Update esp32dev-rf partition

* Fix Weatherstation build

* Fix GFSunInverter build

* Fix esp32dev-ir build

* Fix ble-aws build

* Fix eth builds

* Fix m5Stack missing pins_arduino.h

* Fix build errors for M5 stack/tough, others are upstream issues.

* Fix RTL 433 build - remaining errors are from radolib

* Fix nodemcu build

* fix 2g builds

* Fix serial build

* Fix actuator on off builds

* Fix SSD1306 build - remaining errors are from radiolib

* Fix multiple definition of OTAserver_cert

* Fix nodemcu rf2 build

* Fix ADC builds

* Fix sensor builds

* Fix LORA builds

* Fix multi-receiver builds - remaining errors are in radiolib

* Fix fastled builds

* Fix theegns board builds

* Fix broker builds

* Update fastled - old version failed all-test build

* Fix RN8209 builds

* Fix max temp actuator builds

* Fix PWM builds

* Fix C37 sensor builds

* Fix HTU21 builds

* Fix INA266 builds

* Fix undefined variables in mqtt discovery

* Fix webui build

* Fix fastled invalid pin error

* Fix wifi manual setup builds

* Fix onewire/all-test build - bin too big error remaining

* Fix theengs plug build

* Fix rfbridge builds

* Fix blufi builds

* Fix undefined functions in serial

* Fix preprocessor definition checks

* Set IDF log level to erre

* Add delay in loop to prevent watchdog timeout

* Use xTaskCreateUniveral to support single core processors

* Remove old HTTPUpdate files - upsteam fixed.

* Cleanup and move common declarations to header file

* Use custom partiton table to fix builds where bin is too large

* Update M5StickC - fixs esp32-m5stick-c-ble build

* Revert to Arduino core 2.x for builds with incompatible libs

* Remove "Z" from file names and rename omg_common to TheengsCommon

* Fix gateway name when using MAC with new Arduino core

* Update IDF config to reduce loop task stack use

* Update esp-nimble-cpp version, corrects BLE uppercase ID's

* Update wifi manager to fix watchdog timeout error
2025-05-06 19:35:50 -05:00
DigiH
6e77efb11b "ohdiscovery" back in obsoleteEntities 2025-04-28 17:14:40 +02:00
peshovec
7f542c8347 [RF] ZgatewatPilight: toggle RawEnabled only if Pilight is active receiver (#2159) 2025-03-07 16:48:15 -06:00
Alessandro Staniscia
e683fbbd18 [RF] Proposal fix for issue #2139 + Extra documentation (#2157)
* Refactor announceDeviceTrigger function to improve parameter handling and documentation for optional device information
Fix null pointer references in RFtoMQTTdiscovery and update logging levels for RF signal handling
Refactor RF to MQTT discovery functions for improved clarity and parameter handling

* Update documentation for RF auto discovery and MQTT integration, correcting terminology and enhancing clarity

* Refactor enableRFReceive function to accept parameters for frequency and GPIOs, enhancing flexibility
2025-03-07 16:47:33 -06:00
peshovec
e2ada763ec [RF2] fix json formatted sending, when the active receiver is not RF2 (#2158) 2025-02-04 07:39:26 -06:00
Ryan Powell
d03795d0bf [BT] Update NimBLE-Arduino to 2.2.1 (#2140)
* Update NimBLE-Arduino to 2.2.1

* Add BLE OTA update capability.
2025-02-01 12:21:40 -06:00
DigiH
4bf43f5695 [DISC] Removal of openHAB specific MQTT discovery patch (#2153)
Removal of openHAB specific MQTT discovery patch "ohdisc", as it is no longer required with openHAB 4.3 and later, and with openHAB before 4.3 MQTT discovery never was fully implemented anyway.
2025-01-25 14:14:22 -06:00
peshovec
d837dfd32a [RF] HA: RF modules as sensors. Subscribe topic, according valueAsATopic (#2134) 2025-01-25 12:53:24 -06:00
Florian
cd35520bb0 [BT] Increase again procBLEtask stack size (#2152)
Co-authored-by: Florian <1technophile@users.noreply.github.com>
2025-01-14 07:23:57 -06:00
Florian
bead01347e [BT] Increase procBLEtask stack size to avoid reset when adapting the scan parameters (#2148)
Co-authored-by: Florian <1technophile@users.noreply.github.com>
2025-01-10 13:47:34 -06:00
DigiH
d8c0f0f9c5 [BT] Small typo fix (#2147) 2025-01-07 17:14:49 +01:00
DanEmord
4d6e7f4afa [RF] Allow configuration of CC1101 SPI pins (#2141)
If all CC1101 SPI pins are defined, use them to create the SPI device. Otherwise, default to the platform's SPI.
2025-01-03 09:29:26 -06:00
DigiH
5e1860abd9 [DISC] rtl_433 UV amendments (#2144) 2025-01-02 14:56:18 +01:00
benzok
312b4bcf17 [DISC] Whitelist immediate change (#2133)
* When updating a device from UNKNOWN force a new scan

* update the sensorModel_id and handle if white-list is updated

* Restore unduly deleted documentation
2024-12-18 07:53:32 -06:00
melyux
7fa00dd593 Pilight: Handle when 'value' is not a string (#2129) 2024-12-13 16:42:39 -06:00
DigiH
1b50c8ec02 [WEBUI] Fix for TH1 external probe in display routine (#2127)
Fix for TH1 external probe in display routine only actually showing when connected.
2024-12-05 15:33:28 +01:00
DigiH
2cce06ed4e [DISC] Defined boolean values (#2126)
Defined boolean values in discovery message for cross platform discovery in HA and OH for binary_sensors.
2024-12-04 01:04:56 +01:00
Florian
1f45a7711b [SYS] Avoid queue overloading at start (#2114)
Give one cycle of the loop without starting BT

Co-authored-by: Florian <1technophile@users.noreply.github.com>
2024-11-15 10:37:20 -06:00
Florian
b5adf1f5ac [BROKER] Fix BT start (#2112)
Co-authored-by: Florian <1technophile@users.noreply.github.com>
2024-11-13 07:42:44 -06:00
Florian
3bcfdee712 [BLUFI] Fix SSID return (#2108)
The filter on NTWK_CONNECTED was too restrictive when asking for WiFi status.
Get the WiFi name and BSSID from the WiFi object

Co-authored-by: Florian <1technophile@users.noreply.github.com>
2024-11-12 08:50:24 -06:00
Florian
379312a127 [BLUFI] Enable larger notifications (#2107)
by replacing esp_blufi_send_notify by esp_blufi_send_custom_data

Co-authored-by: Florian <1technophile@users.noreply.github.com>
2024-11-12 08:37:51 -06:00
Florian
fdc7435079 [SYS] Fix capacity to onboard ethernet board without WiFi AP (#2106)
Co-authored-by: Florian <1technophile@users.noreply.github.com>
2024-11-12 08:26:55 -06:00
Florian
f5fe3ed223 [DISC] Remove unnecessary discovery publication (#2105)
They are not anymore needed as we keep the scan and presence parameters always displayed

Co-authored-by: Florian <1technophile@users.noreply.github.com>
2024-11-12 08:26:29 -06:00
Florian
921488359e [BT] Fix BT command triggering a stack trace due to conflict (#2104)
related to the vector between BLE scanning and BLE control

Co-authored-by: Florian <1technophile@users.noreply.github.com>
2024-11-12 08:26:15 -06:00