fix(importer): clamp over-long external text fields so imports do not hard-fail (#15) #37

Open

jmz wants to merge 2 commits from fix/waypoint-description-clamp into main

pull from: fix/waypoint-description-clamp

merge into: public:main

public:main

public:feat/upstream-sync-action

public:deploy

public:feat/immersive-map

public:pr/smart-insertion

public:feat/smart-insertion

public:feat/heatmap-overlay

public:dependabot/npm_and_yarn/web/fast-uri-3.1.4

public:dependabot/npm_and_yarn/docs/fast-uri-3.1.4

public:feature/app

public:dependabot/npm_and_yarn/web/dompurify-3.4.12

public:l10n_feature/app

public:dependabot/npm_and_yarn/docs/svgo-4.0.2

public:dependabot/npm_and_yarn/docs/astrojs/node-11.0.2

public:dependabot/npm_and_yarn/docs/astro-7.1.3

public:l10n_main

public:chore-update-workflows-for-dev

public:fix/komootMimeType

public:feat/pluginRouting

public:immichIntegration

public:dependabot/npm_and_yarn/docs/multi-a1a9a7eb75

public:dev

public:fix/remote-trail-fetch-with-cache

public:feature/ap-instance-actors

public:feat/e2e-playwright

public:golangci-lint

public:github-pages

public:fixTrailEditIndex2

public:trailBuckets

public:pocketbaseCGO

public:federation

Conversation 0 Commits 2 Files changed 2 +155 -4

jmz commented 2026-09-08 12:31:19 +00:00

Owner

Copy link

Problem (fixes #15)

The importer wrote external strings straight into records and app.Save aborts the entire trail import on any field-validation error. Reported case: a GPX waypoint description over the waypoints collection's 5000-char limit (komoot POI descriptions routinely run 5-6.5k) killed the whole trail. The same failure mode applies to the other external text the importer writes — the waypoint name and the trail name/description — so over-long values there abort imports too (confirmed for waypoint name: name: Must be no more than 255 character(s)).

Fix

Add clampTextField(collection, field, value) — clamps to that field's own rune limit (read from the collection, no-op for unbounded/non-text fields), rune-safe with an ellipsis. Apply it to every external text field createWaypoints and ImportTrail write (waypoint name+description, trail name+description). Mirrors the existing photoFiles skip-don't-fail philosophy: no single over-long external value can abort an import.

Tests (test-first)

• TestCreateWaypointsClampsOverlongDescription — 6000-rune description; reproduced the exact abort before the fix.
• TestCreateWaypointsClampsOverlongName — 600-char name; reproduced name: Must be no more than 255 character(s) before the fix.
• TestClampToRuneLimit — table test (empty, under/at/over limit, multibyte, no-limit).
• Full plugins/importer suite + go vet + go build ./... green (CGO_ENABLED=0, go 1.25).

Atomic, upstreamable. AI-assisted, disclosed (Co-Authored-By).

## Problem (fixes #15) The importer wrote external strings straight into records and `app.Save` **aborts the entire trail import** on any field-validation error. Reported case: a GPX waypoint description over the `waypoints` collection's 5000-char limit (komoot POI descriptions routinely run 5-6.5k) killed the whole trail. The **same failure mode** applies to the other external text the importer writes — the waypoint **name** and the trail **name/description** — so over-long values there abort imports too (confirmed for waypoint name: `name: Must be no more than 255 character(s)`). ## Fix Add `clampTextField(collection, field, value)` — clamps to that field's **own** rune limit (read from the collection, no-op for unbounded/non-text fields), rune-safe with an ellipsis. Apply it to every external text field `createWaypoints` and `ImportTrail` write (waypoint name+description, trail name+description). Mirrors the existing `photoFiles` skip-don't-fail philosophy: no single over-long external value can abort an import. ## Tests (test-first) - `TestCreateWaypointsClampsOverlongDescription` — 6000-rune description; reproduced the exact abort before the fix. - `TestCreateWaypointsClampsOverlongName` — 600-char name; reproduced `name: Must be no more than 255 character(s)` before the fix. - `TestClampToRuneLimit` — table test (empty, under/at/over limit, multibyte, no-limit). - Full `plugins/importer` suite + `go vet` + `go build ./...` green (CGO_ENABLED=0, go 1.25). Atomic, upstreamable. AI-assisted, disclosed (Co-Authored-By).

jmz added 1 commit 2026-09-08 12:31:19 +00:00

fix(importer): clamp over-long waypoint descriptions instead of failing the import ... 01e5af2223

A GPX waypoint whose description exceeds the waypoints collection's 5000
character limit made createWaypoints' app.Save return a validation error,
aborting the ENTIRE trail import — the track and every other waypoint were
lost, not just the one over-long POI. komoot highlight/POI descriptions
routinely run 5-6.5k characters, so bulk GPX imports of multi-day routes
failed outright (oss/wanderer #15).

Clamp the description to the collection's own description-field limit (rune-
safe, with an ellipsis) before saving, mirroring how photoFiles already skips
individual bad media rather than failing the whole trail.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

jmz referenced this pull request 2026-09-08 12:31:39 +00:00

Upstream: GPX upload hard-fails when a waypoint description exceeds 5000 chars #15

maximus added 1 commit 2026-09-08 13:44:10 +00:00

fix(importer): clamp all external text fields, not just waypoint description ... 8aa3465565

Generalizes the #15 guard. The importer also wrote the trail name/description
and the waypoint name unclamped, so an over-long value in any of them aborted
the whole import the same way (confirmed for waypoint name:
'name: Must be no more than 255 character(s)').

Add clampTextField(collection, field, value), which clamps to that field's own
rune limit (no-op for unbounded/non-text fields), and apply it to every
external text field createWaypoints and ImportTrail write.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

jmz changed title from fix(importer): clamp over-long waypoint descriptions instead of failing the import to fix(importer): clamp over-long external text fields so imports do not hard-fail (#15) 2026-09-08 13:44:36 +00:00

jmz referenced this pull request 2026-09-08 13:44:37 +00:00

Upstream: GPX upload hard-fails when a waypoint description exceeds 5000 chars #15

Some checks failed

Hide checks

Go / db-test (pull_request) Failing after 2s

Details

This pull request can be merged automatically.

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin fix/waypoint-description-clamp:fix/waypoint-description-clamp

git switch fix/waypoint-description-clamp

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main

git merge --no-ff fix/waypoint-description-clamp

git switch fix/waypoint-description-clamp

git rebase main

git switch main

git merge --ff-only fix/waypoint-description-clamp

git switch fix/waypoint-description-clamp

git rebase main

git switch main

git merge --no-ff fix/waypoint-description-clamp

git switch main

git merge --squash fix/waypoint-description-clamp

git switch main

git merge --ff-only fix/waypoint-description-clamp

git switch main

git merge fix/waypoint-description-clamp

git push origin main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

public/wanderer!37

No description provided.