Commit Graph

125 Commits

Author SHA1 Message Date
Hannes Rüger
e5dcfad3ff feat(parts table): add eda reference prefix and value columns (#1266)
* feat(parts table): add eda reference prefix and value columns

* Use better labels for column names and made it visible as default column selection

---------

Co-authored-by: Jan Böhmer <mail@jan-boehmer.de>
2026-03-01 22:28:01 +01:00
Sebastian Almberg
b9d940ae33 Enhance KiCad integration: API v2, batch EDA editing, field export control (#1241)
* Add stock quantity, datasheet URL, and HTTP caching to KiCad API

- Add Stock field showing total available quantity across all part lots
- Add Storage Location field when parts have stored locations
- Resolve actual datasheet PDF from attachments (by type name, attachment
  name, or first PDF) instead of always linking to Part-DB page
- Keep Part-DB page URL as separate "Part-DB URL" field
- Add ETag and Cache-Control headers to all KiCad API endpoints
- Support conditional requests (If-None-Match) returning 304
- Categories/part lists cached 5 min, part details cached 1 min

* Add KiCadHelper unit tests and fix PDF detection for external URLs

- Add comprehensive KiCadHelperTest with 14 test cases covering:
  - Stock quantity calculation (zero, single lot, multiple lots)
  - Stock exclusion of expired and unknown-quantity lots
  - Storage location display (present, absent, multiple)
  - Datasheet URL resolution by type name, attachment name, PDF extension
  - Datasheet fallback to Part-DB URL when no match
  - "Data sheet" (with space) name variant matching
- Fix PDF extension detection for external attachments (getExtension()
  returns null for external-only attachments, now also parses URL path)

* Fix 304 response body, parse_url safety, and location/stock consistency

- Use empty Response instead of JsonResponse(null) for 304 Not Modified
  to avoid sending "null" as response body
- Guard parse_url() result with is_string() since it can return false
  for malformed URLs
- Move storage location tracking inside the availability check so
  expired and unknown-quantity lots don't contribute locations

* Fix testPartDetailsPart2 to actually test Part 2

The test was requesting /parts/1.json instead of /parts/2.json and had
Part 1's expected data. Now tests Part 2 which inherits EDA info from
its category and footprint, verifying the inheritance behavior.

* Use Symfony's built-in ETag handling for HTTP caching

Replace manual If-None-Match comparison with Response::setEtag() and
Response::isNotModified(), which properly handles ETag quoting, weak
vs strong comparison, and 304 response cleanup. Fixes PHPStan return
type error and CI test failures.

* Add configurable KiCad field export for part parameters

Add a kicad_export checkbox to parameters, allowing users to control
which specifications appear as fields in the KiCad HTTP library API.
Parameters with kicad_export enabled are included using their formatted
value, without overwriting hardcoded fields like description or Stock.

* Add partdb:kicad:populate command for bulk KiCad path assignment

Console command that populates KiCad footprint/symbol paths on Footprint
and Category entities based on name-to-library mappings. Supports dry-run,
force overwrite, and list modes. Includes 130+ footprint mappings and 30+
category symbol mappings for KiCad 9.x standard libraries.

* Add CSV import support for EDA/KiCad fields

Add user-friendly column aliases (kicad_symbol, kicad_footprint,
kicad_reference, kicad_value, eda_exclude_bom, etc.) to the CSV import
system. Users can now bulk-set KiCad symbols, footprints, and other EDA
metadata via CSV/Excel import without knowing the internal dot notation.

* Add batch EDA field editing from parts table

Users can now select multiple parts in any parts table and batch-edit
their EDA/KiCad fields (symbol, footprint, reference prefix, value,
visibility, exclude from BOM/board/sim). Each field has an "Apply"
checkbox so users control exactly which fields are changed.

* Remove unused counter variable in BatchEdaController

* Fix PHPStan errors in PopulateKicadCommand and BatchEdaController

Add @var type annotations for Doctrine repository findAll() calls so
PHPStan can resolve getEdaInfo() on Footprint/Category entities. Fix
array return type for numeric-string keys and add explicit callback to
array_filter to satisfy strict rules.

* Fix batch EDA edit: required validation and pre-populate shared values

- Add required=false to TriStateCheckboxType fields so HTML5 validation
  doesn't force users to check visibility/BOM/board checkboxes
- Pre-populate form fields when all selected parts share the same EDA
  value, so users can see current state before editing

* Add KiCad API v2, orderdetail export control, EDA status indicator, BOM improvements

- Add KiCad API v2 endpoints (/kicad-api/v2) with volatile field support
  for stock and storage location (shown but not saved to schematic)
- Add kicad_export flag to Orderdetail entity for per-supplier SPN control
  (backward compatible: if no flag set, all SPNs exported as before)
- Add EDA completeness indicator column in parts datatable (bolt icon)
- Add ?minimal=true query param for faster category parts loading
- Improve category descriptions (use comment instead of URL when available)
- Improve BOM importer multi-footprint support: merge entries by Part-DB
  part ID when linked, tracking footprint variants in comments
- Fix KiCost manf/manf# fields always present (not conditional on orderdetails)
- Fix duplicate getEdaInfo() call in shouldPartBeVisible
- Consolidate supplier SPN and KiCost field generation into single loop

* Fix kicad_export column default for SQLite compatibility

Add options default to ORM column definition so schema:update
works correctly on SQLite test databases.

* Make EDA status bolt icon clickable to open EDA settings tab

* Fix EDA bolt link to correctly open EDA tab via data-turbo=false

* Add configurable datasheet URL mode for KiCad API

New setting "Datasheet field links to PDF" in KiCad EDA settings.
When enabled (default), the datasheet field resolves to the actual
PDF attachment URL. When disabled, it links to the Part-DB page
(old behavior). Configurable via settings UI or EDA_KICAD_DATASHEET_AS_PDF env var.

* Fix settings crash when upgrading: make datasheetAsPdf nullable

The settings bundle stores values in the database. When upgrading from
a version without datasheetAsPdf, the stored JSON lacks this key,
causing a TypeError when assigning null to a non-nullable bool.
Making it nullable with a fallback in KiCadHelper fixes the upgrade path.

* Add functional tests for KiCad API v2 and batch EDA controller

- KiCadApiV2ControllerTest: root, categories, parts, volatile fields,
  v1 vs v2 comparison, cache headers, 304 conditional request, auth
- BatchEdaControllerTest: page load, empty redirect, form submission

* Fix test failures: correct ids format and anonymous access assertion

* Improve test coverage for BatchEdaController

Add tests for: applying all EDA fields at once, custom redirect URL,
and verifying unchecked fields are skipped.

* Address PR review: rename to eda_visibility, merge migrations, API versioning

Changes based on jbtronics' review of PR #1241:

- Rename kicad_export -> eda_visibility (entities, forms, templates,
  translations, tests) with nullable bool for system default support
- Merge two database migrations into one (Version20260211000000)
- Rename createCachedJsonResponse -> createCacheableJsonResponse
- Change bool $apiV2 -> int $apiVersion with version validation
- EDA visibility field only shown for part parameters, not other entities
- PopulateKicadCommand: check alternative names of footprints/categories
- PopulateKicadCommand: support external JSON mapping file (--mapping-file)
- Ship default mappings JSON at contrib/kicad-populate/default_mappings.json
- Add system-wide defaultEdaVisibility setting in KiCadEDASettings
- Add KiCad HTTP Library v2 spec link in controller docs

* Fix duplicate loadMappingFile method causing PHP fatal error

* Add tests for mapping file and alternative name matching, update populate command docs

Add 5 new tests for PopulateKicadCommand covering:
- Custom mapping file overriding defaults
- Invalid JSON mapping file error handling
- Missing mapping file error handling
- Footprint alternative name matching
- Category alternative name matching

Update contrib README to document --mapping-file option,
alternative name matching, and custom JSON mapping format.

* Split out KiCad API v2 into separate PR as requested by maintainer

Remove v2 controller, tests, and volatile field support from this PR.
The v2 API will be submitted as a separate PR for focused discussion.

* Improve test coverage for KiCadHelper and PopulateKicadCommand

KiCadHelper: Add tests for orderdetail eda_visibility filtering,
backward compatibility when no flags set, manufacturer/KiCost fields,
and parameter with empty name skipping.

PopulateKicadCommand: Add tests for mapping file with both footprints
and categories sections, and mapping file with only categories.

* Load populate Kicad default mappings from json file

* Moved kicad:populate documentation to central docs

* Added introduced column to PartTableColumns to make it configurable in the settings

* Use TristateCheckboxes for parameter and orderdetail types

* Fixed translation keys

* Split up default eda visibility for parameters and purchase infos

---------

Co-authored-by: Jan Böhmer <mail@jan-boehmer.de>
2026-03-01 22:10:13 +01:00
Jan Böhmer
8e0fcdb73b Added some part datatables optimization 2026-02-15 20:07:38 +01:00
Jan Böhmer
e19929249f Mark parts datatables query as read only for some memory optimizations 2026-02-15 19:30:53 +01:00
Jan Böhmer
a962e5e019 Allow to order and filter by GTIN in part tables 2026-02-08 15:51:39 +01:00
web-devinition.de
14a4f1f437 Added custom part status (#1053)
* Benutzerdefinierten Bauteilstatus einführen

* PartCustomStateController hinzufügen

* Umstellung Migrationen bzgl. Multi-Plattform-Support.
Zunächst MySQL, SQLite Statements integrieren.

* Postgre Statements integrieren

* Semikolon in Migration entfernen

* Migration für PartCustomState aktualisieren

* Benutzerdefinierten Bauteilstatus in TableSettings aufnehmen

* PartCustomStateControllerTest: Attribute für PHPUnit-Gruppen umgestellt

* PartCustomState: Mapping für Parameter korrigieren

* PartCustomState: Darstellung und Zuordnung von Anhängen ergänzt

Die Sidebar wurde um die Anzeige des benutzerdefinierten Bauteilstatus erweitert, inklusive Vorschaubild, sofern vorhanden.

* Migrationen zusammenführen

* PartCustomState: Anpassungen bzgl. Tests

* PartCustomStateEndpoint hinzufügen

* Made custom part states plural for consistency with other entity captions

* Fixed phpunit error

* Fixed phpstan issues

---------

Co-authored-by: Marcel Diegelmann <marcel.diegelmann@gmail.com>
Co-authored-by: Jan Böhmer <mail@jan-boehmer.de>
2025-10-27 21:58:16 +01:00
Jan Böhmer
0d49632b92 Refactored constraints, to reuse existing mechanisms 2025-09-21 20:45:18 +02:00
barisgit
cc9d50a8fe Add makefile to help with development setup, change part_ids in bulk import jobs to junction table and implement filtering based on bulk import jobs status and its associated parts' statuses. 2025-08-31 22:17:05 +02:00
Jan Böhmer
442457f11b Merge branch 'master' into settings-bundle 2025-06-15 18:39:49 +02:00
Jan Böhmer
3247a97217 Removed "All" page length option for log tables, as this easily causes an out of memory situation on the server
You can easily have thousands of log entries, making it impossible to view them all at once.

This fixes #835
2025-02-01 23:44:07 +01:00
Jan Böhmer
8750573724 Merge branch 'master' into settings-bundle 2025-01-17 22:06:18 +01:00
Jan Böhmer
01fc6524a4 Added a aggregate function for storelocation sorting to avoid exceptions on Postgres
Actually this was not good on other DB types too, but they just ignored the problems.

This fixes issue #734
2024-11-04 23:46:45 +01:00
Jan Böhmer
0dbf417866 Moved part table default column definiton to settings UI 2024-08-21 22:02:56 +02:00
Jan Böhmer
20f32c7f12 Applied rector suggestions 2024-06-22 00:31:43 +02:00
Jan Böhmer
b4d8d31c2d partUnit column in part datatable now shows the correct data
This fixes issue #630
2024-06-19 23:28:53 +02:00
Jan Böhmer
3f471d0c73 Natsort MPN column in part datatables 2024-06-18 22:34:29 +02:00
Jan Böhmer
4a706ab707 Do not natsort description fields as these can become very long 2024-06-18 22:34:13 +02:00
Jan Böhmer
9db822eabd Use natural sorting for string datatables columns when using postgres
The natural sorting solution is quite portable, so this should be possible for other database types too later
2024-06-17 21:38:16 +02:00
Jan Böhmer
6d1553e8d8 Fixed problem with datatable column sorting on postgresql 2024-06-16 22:52:15 +02:00
Jan Böhmer
c58ff5861d Replaced the non standard IFNULL function which postgres does not know with the COALSCE function 2024-06-13 23:01:53 +02:00
Jan Böhmer
d49bb1f4b5 Removed unused import statements 2024-03-03 20:34:36 +01:00
Jan Böhmer
42e604245c Fixed some inspection issues 2024-03-03 19:57:31 +01:00
Jan Böhmer
04d1e84596 Added a filter to filter for parts which are used in a specific project
Related to issue #516
2024-03-02 19:00:30 +01:00
Jan Böhmer
934acca934 Added an column to part table view to show which projects a part uses.
Fixes issue #516
2024-03-02 18:48:31 +01:00
Jan Böhmer
a72a61091a Only add the amountSum select for part tables if it is really required 2024-02-29 23:35:05 +01:00
Jan Böhmer
c635637c00 Fixed problem that tables could not be sorted by manufacturers, categories, etc. 2024-02-29 23:13:17 +01:00
Jan Böhmer
2e5b2c8b7d Removed amountSum subquery from the detail query, as its result is not used anywhere 2024-02-25 01:46:35 +01:00
Jan Böhmer
fbd095ab50 Only add the joins to the parts table filter query if they are really required
This should improve the performance of the queries slightly
2024-02-25 01:40:25 +01:00
Jan Böhmer
b176cb1ae1 Use a much faster query to retrieve the total number of rows in datatables
This should increase performance on part tables
2024-02-25 00:58:26 +01:00
Jan Böhmer
83ad99215f Added optional "stocked amount" and storage locations columns for the BOM list
This fixes issue #429
2023-11-19 22:13:25 +01:00
Jan Böhmer
1becde537b Fixed test issues 2023-10-08 21:54:51 +02:00
Jan Böhmer
3c5a96ba5c Merge branch 'part-table-columns-config' 2023-10-08 21:50:43 +02:00
Jan Böhmer
470df57f58 Removed useless permissions checks, as the permissions are now always granted automatically, if the user has read access to parts 2023-10-08 21:36:05 +02:00
Jan Böhmer
185c88fa3e Removed now useless visibility options from PartsDataTable
The visibility is now configured by the env variable, so this is useless.
2023-10-08 21:32:57 +02:00
Jan Böhmer
79262972aa Renamed config env to TABLE_PARTS_DEFAULT_COLUMNS and updated documentation 2023-10-08 21:28:37 +02:00
Jan Böhmer
1369091b90 Moved column sorting and visibility logic to its own (non-shared) helper service 2023-10-08 21:07:22 +02:00
Jan Böhmer
1b276c5eae Fixed typo in name of TwoStepORMAdapter 2023-10-06 13:08:46 +02:00
misaz
0753b7137f fixed tab/spaces in PartsDataTable 2023-10-04 21:30:04 +02:00
misaz
9d54001f89 added support for configuring columns in part table which are enabled by default as well as their order. 2023-10-04 21:14:57 +02:00
Jan Böhmer
0af5a58dbe Renamed Storelocation entity to StorageLocation 2023-09-04 22:57:40 +02:00
Jan Böhmer
62b1e33616 Fixed field ordering on SQLite 2023-07-29 16:42:27 +02:00
Jan Böhmer
4c4501073c Use a two step approach to retrieve part tables
This allows us to combine the fast filtering with the fetch joins, which should speed up the datatables a lot.
2023-07-23 21:48:38 +02:00
Jan Böhmer
d59b8817c3 Do not use fetch join, as even with the N+1 problem the queries are faster than with the very complex and slow expressions needed for the fetch Join pagination 2023-07-22 23:51:06 +02:00
Jan Böhmer
9e3cb4d694 Use enum for manufacturing status 2023-07-08 23:49:47 +02:00
Jan Böhmer
b941b97eee Show full paths of elements on hover in part tables
Related to discussion #312
2023-07-08 19:02:43 +02:00
Jan Böhmer
5629215ce4 Use imports instead of FQNs 2023-06-11 15:00:28 +02:00
Jan Böhmer
7ee01d9a05 Applied rector with PHP8.1 migration rules 2023-06-11 14:17:19 +02:00
Jan Böhmer
a43af180a7 Applied rector rules up to symfony 6.2 2023-05-28 01:21:05 +02:00
Jan Böhmer
334d81db08 Mark that amount is unknown in part tables and info page
Fixes issue #282
2023-04-29 22:33:46 +02:00
Jan Böhmer
e8ec536a5a Allow to order by storelocation column
Related to discussion #269
2023-04-23 23:38:59 +02:00