Compare commits

..

10 Commits

Author SHA1 Message Date
Cursor Agent 80189ba449 fix(unstable): correct Parser.Range for non-suffix highlights
Parser.Range used len(data)-len(highlight), which only matches suffix
slices. Single-byte highlights like b[0:1] are subslices of the
remaining buffer, so the wrong offset pointed at the end of the
document (issue #1047). Use pointer-based subslice offset like
wrapDecodeError.

Add regression tests for unstable.Parser and toml.Unmarshal error
positions and human-readable output.

Co-authored-by: Thomas Pelletier <thomas@pelletier.dev>
2026-04-12 12:26:03 +00:00
Thomas Pelletier f36a3ece9e Reduce marshal and unmarshal overhead (#1044)
* Reduce marshal and unmarshal overhead

Targeted optimizations to reduce performance overhead introduced by
recent feature additions and the unsafe removal.

Unmarshal:
- parseKeyval: access the node directly in the builder's slice to set
  Raw, bypassing NodeAt which triggers a GC write barrier for the
  nodes-pointer on every key-value expression.
- Iterator.Next: cache the *nodes slice dereference in a local variable
  to avoid repeated pointer-to-slice indirection in the hot loop.

Marshal:
- Guard shouldOmitZero calls with an inlineable options.omitzero check.
  shouldOmitZero has inlining cost 1145 (budget 80), so avoiding the
  function call when omitzero is not set removes per-field overhead.
- Inline the isNil check in encodeMap. isNil has inlining cost 93
  (budget 80), so expanding it at the single hot call site avoids
  per-map-entry function call overhead.

Update README benchmarks.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 11:08:39 +00:00
Thomas Pelletier 77f3862df4 Fix benchmark script replacing internal package imports (#1042)
* Fix benchmark script replacing internal package imports

The sed command in bench() was replacing all occurrences of the go-toml
module path, including sub-package imports like internal/assert. This
caused the BurntSushi/toml benchmark to fail because it tried to import
github.com/BurntSushi/toml/internal/assert which doesn't exist.

Fix by anchoring the sed pattern to only match the import path when
followed by a closing quote, preserving internal package imports.

Also add a guard in the benchstathtml Python script to give a clear
error instead of an IndexError when no benchmark results are available.

https://claude.ai/code/session_016JGASo49PeFSfCaDxvrGFE

* Update benchmark results in README

https://claude.ai/code/session_016JGASo49PeFSfCaDxvrGFE

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-23 22:00:18 -04:00
Thomas Pelletier 16b1ef5508 Fix parser error pointing to wrong line when last line has no trailing newline (#1041)
When parsing a key without '=' at EOF (e.g., "a = 1\nb = 2\nc"), the
error highlight was an empty slice, causing subsliceOffset to return 0
and the error to point at line 1 instead of line 3. Pass the consumed
key bytes as the highlight instead of the empty remainder.

Fixes #1032

https://claude.ai/code/session_01UWv8pyc8P1ktAPfHpveixj

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-23 21:34:12 -04:00
dependabot[bot] e14bde7c1d build(deps): bump docker/login-action from 3 to 4 (#1039)
Bumps [docker/login-action](https://github.com/docker/login-action) from 3 to 4.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 21:04:21 -04:00
dependabot[bot] 4b1ff01eb3 build(deps): bump docker/setup-buildx-action from 3 to 4 (#1040)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 21:04:11 -04:00
Thomas Pelletier 048a25f0f2 Go 1.26 (#1030)
* ci(release): drop unsupported windows/arm targets
2026-03-03 01:29:35 -05:00
dependabot[bot] b3575580f9 build(deps): bump goreleaser/goreleaser-action from 6 to 7 (#1035) 2026-03-03 00:47:47 -05:00
dependabot[bot] a0be52f4c1 build(deps): bump actions/upload-artifact from 6 to 7 (#1036) 2026-03-03 00:47:35 -05:00
Thomas Pelletier 316bfc66a4 Support Unmarshaler interface for tables and array tables (#1027)
Fixes #873

Extend the unstable.Unmarshaler interface support to work with tables
and array tables, not just single values.

When a type implementing unstable.Unmarshaler is the target of a table
(e.g., [table] or [[array]]), the UnmarshalTOML method receives a
synthetic InlineTable node containing all the key-value pairs belonging
to that table.

Key changes:
- Add handleKeyValuesUnmarshaler to collect and process table content
- Add copyExpressionNodes to deep-copy AST nodes for synthetic tables
- Add helper functions in unstable/ast.go for node manipulation
- Update documentation for EnableUnmarshalerInterface
- Add comprehensive tests for table and array table unmarshaling

* Implement bytes-based Unmarshaler interface for tables and arrays (#873)

This change brings back support for the unstable.Unmarshaler interface
for tables and array tables, addressing issue #873.

Key changes:
- Changed UnmarshalTOML signature from (*Node) to ([]byte) to provide
  raw TOML bytes instead of AST nodes
- Added RawMessage type (similar to json.RawMessage) for capturing raw
  TOML bytes for later processing
- Updated handleKeyValuesUnmarshaler to reconstruct key-value lines
  from the parsed keys and raw value bytes
- Added support for slice types implementing Unmarshaler (e.g., RawMessage)
- Removed unused AST helper functions from unstable/ast.go

The bytes-based interface allows users to:
- Get raw TOML bytes for custom parsing
- Delay TOML decoding using RawMessage
- Implement custom unmarshaling logic for complex types

Tests added for:
- Table unmarshaler with various scenarios
- Array table unmarshaler
- Split tables (same parent defined in multiple places)
- RawMessage usage
- Nested tables and mixed regular fields

* Fix lint issues and improve test coverage for Unmarshaler interface

- Apply De Morgan's law in keyNeedsQuoting to satisfy staticcheck QF1001
- Remove unused splitTableUnmarshaler type from test
- Fix unused parameter lint warning in errorUnmarshaler873
- Add test for quoted keys that need special handling
- Add test for error propagation from UnmarshalTOML
- Update customTable873 parser to handle quoted keys properly

Coverage improved:
- handleKeyValuesUnmarshaler: 80.0% -> 93.3%
- keyNeedsQuoting: 66.7% -> 83.3%
- Overall main package: 97.2% -> 97.5%

* Add test for dotted keys to improve coverage

Add TestIssue873_DottedKeys to test dotted key handling (e.g., sub.key = value)
in the Unmarshaler interface. This improves coverage for handleKeyValuesUnmarshaler
from 93.3% to 96.7%.

* Add double pointer test to achieve 100% coverage for handleKeyValues

Add TestIssue873_DoublePointerUnmarshaler to test pointer-to-pointer
to Unmarshaler types. This covers the pointer dereferencing loop in
handleKeyValues, bringing its coverage from 88% to 100%.

Total coverage: 97.4%

* Add Example tests and fix raw value extraction for boolean types

Add two godoc Example tests:
- ExampleDecoder_EnableUnmarshalerInterface_dynamicConfig: shows dynamic
  unmarshaling based on a type field
- ExampleDecoder_EnableUnmarshalerInterface_rawMessage: demonstrates
  RawMessage usage for deferred parsing

Fix handleKeyValuesUnmarshaler to handle values where Raw.Length == 0
(like boolean types) by using value.Data as fallback.

* Preserve original formatting in Unmarshaler by using raw byte ranges

Instead of reconstructing key-value lines from parsed components, now
uses the original raw bytes from the document. This preserves:
- Whitespace around '=' (e.g., "key   =   value")
- String quoting style (basic vs literal)
- Number formats (hex, octal, binary)
- Inline table formatting

Changes:
- Add Raw range tracking to KeyValue expressions in parseKeyval
- Update handleKeyValuesUnmarshaler to use expr.Raw directly
- Remove keyNeedsQuoting helper (no longer needed)
- Add TestIssue873_FormattingPreservation test
- Update expected output in ExampleParser_comments

* Prevent test matrix from canceling on first failure

Add fail-fast: false to the test workflow strategy so that all
OS/Go version combinations continue running even if one fails.
This provides better visibility into which specific combinations
have issues.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-11 09:57:23 -05:00
16 changed files with 165 additions and 324 deletions
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
dry-run: false dry-run: false
language: go language: go
- name: Upload Crash - name: Upload Crash
uses: actions/upload-artifact@v6 uses: actions/upload-artifact@v7
if: failure() && steps.build.outcome == 'success' if: failure() && steps.build.outcome == 'success'
with: with:
name: artifacts name: artifacts
+1 -1
View File
@@ -15,6 +15,6 @@ jobs:
- name: Setup go - name: Setup go
uses: actions/setup-go@v6 uses: actions/setup-go@v6
with: with:
go-version: "1.25" go-version: "1.26"
- name: Run tests with coverage - name: Run tests with coverage
run: ./ci.sh coverage -d "${GITHUB_BASE_REF-HEAD}" run: ./ci.sh coverage -d "${GITHUB_BASE_REF-HEAD}"
+2 -2
View File
@@ -20,7 +20,7 @@ jobs:
fetch-depth: 0 fetch-depth: 0
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v4
- name: Run Go versions compatibility test - name: Run Go versions compatibility test
run: | run: |
@@ -28,7 +28,7 @@ jobs:
./test-go-versions.sh --output ./test-results $VERSIONS ./test-go-versions.sh --output ./test-results $VERSIONS
- name: Upload test results - name: Upload test results
uses: actions/upload-artifact@v6 uses: actions/upload-artifact@v7
with: with:
name: go-versions-test-results name: go-versions-test-results
path: | path: |
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
- name: Setup go - name: Setup go
uses: actions/setup-go@v6 uses: actions/setup-go@v6
with: with:
go-version: "1.24" go-version: "1.26"
- name: Run golangci-lint - name: Run golangci-lint
uses: golangci/golangci-lint-action@v9 uses: golangci/golangci-lint-action@v9
with: with:
+3 -3
View File
@@ -22,15 +22,15 @@ jobs:
- name: Set up Go - name: Set up Go
uses: actions/setup-go@v6 uses: actions/setup-go@v6
with: with:
go-version: "1.25" go-version: "1.26"
- name: Login to GitHub Container Registry - name: Login to GitHub Container Registry
uses: docker/login-action@v3 uses: docker/login-action@v4
with: with:
registry: ghcr.io registry: ghcr.io
username: ${{ github.actor }} username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }} password: ${{ secrets.GITHUB_TOKEN }}
- name: Run GoReleaser - name: Run GoReleaser
uses: goreleaser/goreleaser-action@v6 uses: goreleaser/goreleaser-action@v7
with: with:
distribution: goreleaser distribution: goreleaser
version: '~> v2' version: '~> v2'
+2 -1
View File
@@ -10,9 +10,10 @@ on:
jobs: jobs:
build: build:
strategy: strategy:
fail-fast: false
matrix: matrix:
os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest', 'macos-14' ] os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest', 'macos-14' ]
go: [ '1.24', '1.25' ] go: [ '1.25', '1.26' ]
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
name: ${{ matrix.go }}/${{ matrix.os }} name: ${{ matrix.go }}/${{ matrix.os }}
steps: steps:
-3
View File
@@ -22,7 +22,6 @@ builds:
- linux_riscv64 - linux_riscv64
- windows_amd64 - windows_amd64
- windows_arm64 - windows_arm64
- windows_arm
- darwin_amd64 - darwin_amd64
- darwin_arm64 - darwin_arm64
- id: tomljson - id: tomljson
@@ -42,7 +41,6 @@ builds:
- linux_riscv64 - linux_riscv64
- windows_amd64 - windows_amd64
- windows_arm64 - windows_arm64
- windows_arm
- darwin_amd64 - darwin_amd64
- darwin_arm64 - darwin_arm64
- id: jsontoml - id: jsontoml
@@ -62,7 +60,6 @@ builds:
- linux_arm - linux_arm
- windows_amd64 - windows_amd64
- windows_arm64 - windows_arm64
- windows_arm
- darwin_amd64 - darwin_amd64
- darwin_arm64 - darwin_arm64
universal_binaries: universal_binaries:
+17 -17
View File
@@ -239,12 +239,12 @@ Execution time speedup compared to other Go TOML libraries:
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr> <tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead> </thead>
<tbody> <tbody>
<tr><td>Marshal/HugoFrontMatter-2</td><td>1.9x</td><td>2.2x</td></tr> <tr><td>Marshal/HugoFrontMatter-2</td><td>2.1x</td><td>2.0x</td></tr>
<tr><td>Marshal/ReferenceFile/map-2</td><td>1.7x</td><td>2.1x</td></tr> <tr><td>Marshal/ReferenceFile/map-2</td><td>2.0x</td><td>2.0x</td></tr>
<tr><td>Marshal/ReferenceFile/struct-2</td><td>2.2x</td><td>3.0x</td></tr> <tr><td>Marshal/ReferenceFile/struct-2</td><td>2.3x</td><td>2.5x</td></tr>
<tr><td>Unmarshal/HugoFrontMatter-2</td><td>2.9x</td><td>2.7x</td></tr> <tr><td>Unmarshal/HugoFrontMatter-2</td><td>3.3x</td><td>2.8x</td></tr>
<tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.6x</td><td>2.7x</td></tr> <tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.9x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.6x</td><td>5.1x</td></tr> <tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.8x</td><td>5.0x</td></tr>
</tbody> </tbody>
</table> </table>
<details><summary>See more</summary> <details><summary>See more</summary>
@@ -257,17 +257,17 @@ provided for completeness.</p>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr> <tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead> </thead>
<tbody> <tbody>
<tr><td>Marshal/SimpleDocument/map-2</td><td>1.8x</td><td>2.7x</td></tr> <tr><td>Marshal/SimpleDocument/map-2</td><td>2.0x</td><td>2.9x</td></tr>
<tr><td>Marshal/SimpleDocument/struct-2</td><td>2.7x</td><td>3.8x</td></tr> <tr><td>Marshal/SimpleDocument/struct-2</td><td>2.5x</td><td>3.6x</td></tr>
<tr><td>Unmarshal/SimpleDocument/map-2</td><td>3.8x</td><td>3.0x</td></tr> <tr><td>Unmarshal/SimpleDocument/map-2</td><td>4.2x</td><td>3.4x</td></tr>
<tr><td>Unmarshal/SimpleDocument/struct-2</td><td>5.6x</td><td>4.1x</td></tr> <tr><td>Unmarshal/SimpleDocument/struct-2</td><td>5.9x</td><td>4.4x</td></tr>
<tr><td>UnmarshalDataset/example-2</td><td>3.0x</td><td>3.2x</td></tr> <tr><td>UnmarshalDataset/example-2</td><td>3.2x</td><td>2.9x</td></tr>
<tr><td>UnmarshalDataset/code-2</td><td>2.3x</td><td>2.9x</td></tr> <tr><td>UnmarshalDataset/code-2</td><td>2.4x</td><td>2.8x</td></tr>
<tr><td>UnmarshalDataset/twitter-2</td><td>2.6x</td><td>2.7x</td></tr> <tr><td>UnmarshalDataset/twitter-2</td><td>2.7x</td><td>2.5x</td></tr>
<tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.2x</td><td>2.3x</td></tr> <tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.3x</td><td>2.3x</td></tr>
<tr><td>UnmarshalDataset/canada-2</td><td>1.8x</td><td>1.5x</td></tr> <tr><td>UnmarshalDataset/canada-2</td><td>1.9x</td><td>1.5x</td></tr>
<tr><td>UnmarshalDataset/config-2</td><td>4.1x</td><td>2.9x</td></tr> <tr><td>UnmarshalDataset/config-2</td><td>5.4x</td><td>3.0x</td></tr>
<tr><td>geomean</td><td>2.7x</td><td>2.8x</td></tr> <tr><td>geomean</td><td>2.9x</td><td>2.8x</td></tr>
</tbody> </tbody>
</table> </table>
<p>This table can be generated with <code>./ci.sh benchmark -a -html</code>.</p> <p>This table can be generated with <code>./ci.sh benchmark -a -html</code>.</p>
+6 -1
View File
@@ -147,7 +147,7 @@ bench() {
pushd "$dir" pushd "$dir"
if [ "${replace}" != "" ]; then if [ "${replace}" != "" ]; then
find ./benchmark/ -iname '*.go' -exec sed -i -E "s|github.com/pelletier/go-toml/v2|${replace}|g" {} \; find ./benchmark/ -iname '*.go' -exec sed -i -E "s|github.com/pelletier/go-toml/v2\"|${replace}\"|g" {} \;
go get "${replace}" go get "${replace}"
fi fi
@@ -195,6 +195,11 @@ for line in reversed(lines[2:]):
"%.1fx" % (float(line[3])/v2), # v1 "%.1fx" % (float(line[3])/v2), # v1
"%.1fx" % (float(line[7])/v2), # bs "%.1fx" % (float(line[7])/v2), # bs
]) ])
if not results:
print("No benchmark results to display.", file=sys.stderr)
sys.exit(1)
# move geomean to the end # move geomean to the end
results.append(results[0]) results.append(results[0])
del results[0] del results[0]
+38
View File
@@ -202,6 +202,38 @@ func TestDecodeError_Accessors(t *testing.T) {
assert.Equal(t, "bar", e.String()) assert.Equal(t, "bar", e.String())
} }
func TestDecodeError_InvalidKeyStartAfterComment(t *testing.T) {
// Regression for https://github.com/pelletier/go-toml/issues/1047: the "="
// that starts an invalid keyval must be reported on line 2, column 1, with
// the human-readable context pointing at that byte (not the document end).
doc := "# comment\n= \"value\""
var v map[string]any
err := Unmarshal([]byte(doc), &v)
if err == nil {
t.Fatal("expected an error")
}
var derr *DecodeError
if !errors.As(err, &derr) {
t.Fatalf("expected *DecodeError, got %T", err)
}
row, col := derr.Position()
if row != 2 || col != 1 {
t.Errorf("Position(): got row %d col %d, want row 2 col 1", row, col)
}
human := derr.String()
if !strings.Contains(human, `2| = "value"`) {
t.Errorf("human output should show the error line; got:\n%s", human)
}
// Caret line uses line-number column width padding; only the "| ~" part is stable here.
if !strings.Contains(human, "| ~ invalid character at start of key") {
t.Errorf("human output should underline '=' and include the parser message; got:\n%s", human)
}
}
func TestDecodeError_DuplicateContent(t *testing.T) { func TestDecodeError_DuplicateContent(t *testing.T) {
// This test verifies that when the same content appears multiple times // This test verifies that when the same content appears multiple times
// in the document, the error correctly points to the actual location // in the document, the error correctly points to the actual location
@@ -259,6 +291,12 @@ func TestDecodeError_Position(t *testing.T) {
expectedRow: 3, expectedRow: 3,
minCol: 5, minCol: 5,
}, },
{
name: "missing equals on last line without trailing newline",
doc: "a = 1\nb = 2\nc",
expectedRow: 3,
minCol: 1,
},
} }
for _, e := range examples { for _, e := range examples {
+12 -9
View File
@@ -704,15 +704,18 @@ func (enc *Encoder) encodeMap(b []byte, ctx encoderCtx, v reflect.Value) ([]byte
for iter.Next() { for iter.Next() {
v := iter.Value() v := iter.Value()
if isNil(v) { // Handle nil values: convert nil pointers to zero value,
// For nil pointers, convert to zero value of the element type. // skip nil interfaces and nil maps.
// This allows round-trip marshaling of maps with nil pointer values. switch v.Kind() {
// For nil interfaces and nil maps, skip since we can't derive a type. case reflect.Ptr:
if v.Kind() == reflect.Ptr { if v.IsNil() {
v = reflect.Zero(v.Type().Elem()) v = reflect.Zero(v.Type().Elem())
} else { }
case reflect.Interface, reflect.Map:
if v.IsNil() {
continue continue
} }
default:
} }
k, err := enc.keyToString(iter.Key()) k, err := enc.keyToString(iter.Key())
@@ -936,7 +939,7 @@ func (enc *Encoder) encodeTable(b []byte, ctx encoderCtx, t table) ([]byte, erro
if shouldOmitEmpty(kv.Options, kv.Value) { if shouldOmitEmpty(kv.Options, kv.Value) {
continue continue
} }
if shouldOmitZero(kv.Options, kv.Value) { if kv.Options.omitzero && shouldOmitZero(kv.Options, kv.Value) {
continue continue
} }
hasNonEmptyKV = true hasNonEmptyKV = true
@@ -958,7 +961,7 @@ func (enc *Encoder) encodeTable(b []byte, ctx encoderCtx, t table) ([]byte, erro
if shouldOmitEmpty(table.Options, table.Value) { if shouldOmitEmpty(table.Options, table.Value) {
continue continue
} }
if shouldOmitZero(table.Options, table.Value) { if table.Options.omitzero && shouldOmitZero(table.Options, table.Value) {
continue continue
} }
if first { if first {
@@ -995,7 +998,7 @@ func (enc *Encoder) encodeTableInline(b []byte, ctx encoderCtx, t table) ([]byte
if shouldOmitEmpty(kv.Options, kv.Value) { if shouldOmitEmpty(kv.Options, kv.Value) {
continue continue
} }
if shouldOmitZero(kv.Options, kv.Value) { if kv.Options.omitzero && shouldOmitZero(kv.Options, kv.Value) {
continue continue
} }
+5 -4
View File
@@ -9,7 +9,7 @@ YELLOW='\033[1;33m'
BLUE='\033[0;34m' BLUE='\033[0;34m'
NC='\033[0m' # No Color NC='\033[0m' # No Color
# Go versions to test (1.11 through 1.25) # Go versions to test (1.11 through 1.26)
GO_VERSIONS=( GO_VERSIONS=(
"1.11" "1.11"
"1.12" "1.12"
@@ -26,6 +26,7 @@ GO_VERSIONS=(
"1.23" "1.23"
"1.24" "1.24"
"1.25" "1.25"
"1.26"
) )
# Default values # Default values
@@ -64,7 +65,7 @@ EXAMPLES:
$0 # Test all Go versions in parallel $0 # Test all Go versions in parallel
$0 --sequential # Test all Go versions sequentially $0 --sequential # Test all Go versions sequentially
$0 1.21 1.22 1.23 # Test specific versions $0 1.21 1.22 1.23 # Test specific versions
$0 --verbose --output ./results 1.24 1.25 # Verbose output to custom directory $0 --verbose --output ./results 1.25 1.26 # Verbose output to custom directory
EXIT CODES: EXIT CODES:
0 Recent Go versions pass (good compatibility) 0 Recent Go versions pass (good compatibility)
@@ -136,8 +137,8 @@ fi
# Validate Go versions # Validate Go versions
for version in "${GO_VERSIONS[@]}"; do for version in "${GO_VERSIONS[@]}"; do
if ! [[ "$version" =~ ^1\.(1[1-9]|2[0-5])$ ]]; then if ! [[ "$version" =~ ^1\.(1[1-9]|2[0-6])$ ]]; then
log_error "Invalid Go version: $version. Supported versions: 1.11-1.25" log_error "Invalid Go version: $version. Supported versions: 1.11-1.26"
exit 1 exit 1
fi fi
done done
+1 -257
View File
@@ -4554,266 +4554,10 @@ func (c *customTable873) UnmarshalTOML(data []byte) error {
c.Keys = append(c.Keys, key) c.Keys = append(c.Keys, key)
c.Values[key] = string(valueBytes) c.Values[key] = string(valueBytes)
} }
return nil return nil
} }
func TestIssue873_TableUnmarshaler(t *testing.T) {
type Config struct {
Section customTable873 `toml:"section"`
}
doc := `
[section]
key1 = "value1"
key2 = "value2"
key3 = "value3"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, []string{"key1", "key2", "key3"}, cfg.Section.Keys)
assert.Equal(t, "value1", cfg.Section.Values["key1"])
assert.Equal(t, "value2", cfg.Section.Values["key2"])
assert.Equal(t, "value3", cfg.Section.Values["key3"])
}
func TestIssue873_TableUnmarshaler_EmptyTable(t *testing.T) {
type Config struct {
Section customTable873 `toml:"section"`
}
doc := `
[section]
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, []string{}, cfg.Section.Keys)
}
func TestIssue873_ArrayTableUnmarshaler(t *testing.T) {
type Config struct {
Items []customTable873 `toml:"items"`
}
doc := `
[[items]]
name = "first"
id = "1"
[[items]]
name = "second"
id = "2"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, 2, len(cfg.Items))
assert.Equal(t, []string{"name", "id"}, cfg.Items[0].Keys)
assert.Equal(t, "first", cfg.Items[0].Values["name"])
assert.Equal(t, "1", cfg.Items[0].Values["id"])
assert.Equal(t, []string{"name", "id"}, cfg.Items[1].Keys)
assert.Equal(t, "second", cfg.Items[1].Values["name"])
assert.Equal(t, "2", cfg.Items[1].Values["id"])
}
func TestIssue873_NestedTableUnmarshaler(t *testing.T) {
type Config struct {
Outer struct {
Inner customTable873 `toml:"inner"`
} `toml:"outer"`
}
doc := `
[outer.inner]
a = "A"
b = "B"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, []string{"a", "b"}, cfg.Outer.Inner.Keys)
assert.Equal(t, "A", cfg.Outer.Inner.Values["a"])
assert.Equal(t, "B", cfg.Outer.Inner.Values["b"])
}
func TestIssue873_TableUnmarshaler_MultipleTables(t *testing.T) {
type Config struct {
First customTable873 `toml:"first"`
Second customTable873 `toml:"second"`
}
doc := `
[first]
key1 = "value1"
[second]
key2 = "value2"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, []string{"key1"}, cfg.First.Keys)
assert.Equal(t, "value1", cfg.First.Values["key1"])
assert.Equal(t, []string{"key2"}, cfg.Second.Keys)
assert.Equal(t, "value2", cfg.Second.Values["key2"])
}
// Test that regular struct fields still work alongside Unmarshaler tables
func TestIssue873_MixedWithRegularFields(t *testing.T) {
type Config struct {
Name string `toml:"name"`
Section customTable873 `toml:"section"`
Count int `toml:"count"`
}
doc := `
name = "test"
count = 42
[section]
foo = "bar"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, "test", cfg.Name)
assert.Equal(t, 42, cfg.Count)
assert.Equal(t, []string{"foo"}, cfg.Section.Keys)
assert.Equal(t, "bar", cfg.Section.Values["foo"])
}
// Test that pointer to Unmarshaler type works
func TestIssue873_PointerToUnmarshaler(t *testing.T) {
type Config struct {
Section *customTable873 `toml:"section"`
}
doc := `
[section]
hello = "world"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.True(t, cfg.Section != nil)
assert.Equal(t, []string{"hello"}, cfg.Section.Keys)
assert.Equal(t, "world", cfg.Section.Values["hello"])
}
// Test table with sub-tables defined separately
func TestIssue873_TableWithSubTables(t *testing.T) {
type Config struct {
Parent customTable873 `toml:"parent"`
}
doc := `
[parent]
name = "root"
[parent.child]
name = "nested"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
// The parent should only get the keys directly under [parent],
// not the [parent.child] sub-table
assert.NoError(t, err)
assert.Equal(t, []string{"name"}, cfg.Parent.Keys)
assert.Equal(t, "root", cfg.Parent.Values["name"])
}
// Test for issue #994 follow-up - tables defined piece-wise
// This addresses the maintainer's comment: "It doesn't deal with tables defined piece-wise yet"
func TestIssue994_TablesPieceWise(t *testing.T) {
// Test with piece-wise table definition (using [table] syntax)
// The customTable873 type captures key-value pairs in order,
// which is useful for use cases like maintaining map ordering
doc := `
[section]
first = "1"
second = "2"
third = "3"
`
type Config struct {
Section customTable873 `toml:"section"`
}
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
// Verify ordering is preserved (keys are collected in document order)
assert.Equal(t, []string{"first", "second", "third"}, cfg.Section.Keys)
assert.Equal(t, "1", cfg.Section.Values["first"])
assert.Equal(t, "2", cfg.Section.Values["second"])
assert.Equal(t, "3", cfg.Section.Values["third"])
}
// Test root-level struct with tables - combines #994 fix with #873 enhancement
func TestIssue994_RootWithTables(t *testing.T) {
type rootDoc struct {
Tables []customTable873 `toml:"tables"`
}
doc := `
[[tables]]
name = "first"
value = "one"
[[tables]]
name = "second"
value = "two"
`
var d rootDoc
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&d)
assert.NoError(t, err)
assert.Equal(t, 2, len(d.Tables))
assert.Equal(t, "first", d.Tables[0].Values["name"])
assert.Equal(t, "one", d.Tables[0].Values["value"])
assert.Equal(t, "second", d.Tables[1].Values["name"])
assert.Equal(t, "two", d.Tables[1].Values["value"])
}
// Test for split tables - when the same parent table is defined in multiple places // Test for split tables - when the same parent table is defined in multiple places
// This is a key requirement for issue #873: if type A implements Unmarshaler, // This is a key requirement for issue #873: if type A implements Unmarshaler,
// and [a.b] and [a.d] are defined with another table [x] in between, // and [a.b] and [a.d] are defined with another table [x] in between,
+7 -3
View File
@@ -28,12 +28,16 @@ func (c *Iterator) Next() bool {
if c.nodes == nil { if c.nodes == nil {
return false return false
} }
nodes := *c.nodes
if !c.started { if !c.started {
c.started = true c.started = true
} else if c.idx >= 0 { } else {
c.idx = (*c.nodes)[c.idx].next idx := c.idx
if idx >= 0 && int(idx) < len(nodes) {
c.idx = nodes[idx].next
} }
return c.idx >= 0 && int(c.idx) < len(*c.nodes) }
return c.idx >= 0 && int(c.idx) < len(nodes)
} }
// IsLast returns true if the current node of the iterator is the last // IsLast returns true if the current node of the iterator is the last
+22 -10
View File
@@ -3,6 +3,7 @@ package unstable
import ( import (
"bytes" "bytes"
"fmt" "fmt"
"reflect"
"unicode" "unicode"
"github.com/pelletier/go-toml/v2/internal/characters" "github.com/pelletier/go-toml/v2/internal/characters"
@@ -69,7 +70,7 @@ func (p *Parser) Data() []byte {
// panics. // panics.
func (p *Parser) Range(b []byte) Range { func (p *Parser) Range(b []byte) Range {
return Range{ return Range{
Offset: uint32(p.subsliceOffset(b)), //nolint:gosec // TOML documents are small Offset: uint32(subsliceOffset(p.data, b)), //nolint:gosec // TOML documents are small
Length: uint32(len(b)), //nolint:gosec // TOML documents are small Length: uint32(len(b)), //nolint:gosec // TOML documents are small
} }
} }
@@ -82,11 +83,21 @@ func (p *Parser) rangeOfToken(token, rest []byte) Range {
return Range{Offset: uint32(offset), Length: uint32(len(token))} //nolint:gosec // TOML documents are small return Range{Offset: uint32(offset), Length: uint32(len(token))} //nolint:gosec // TOML documents are small
} }
// subsliceOffset returns the byte offset of subslice b within p.data. // subsliceOffset returns the byte offset of subslice b within data.
// b must be a suffix (tail) of p.data. // b must share the same backing array as data (any subslice of data).
func (p *Parser) subsliceOffset(b []byte) int { func subsliceOffset(data, b []byte) int {
// b is a suffix of p.data, so its offset is len(p.data) - len(b) if len(b) == 0 {
return len(p.data) - len(b) return 0
}
dataPtr := reflect.ValueOf(data).Pointer()
bPtr := reflect.ValueOf(b).Pointer()
offset := int(bPtr - dataPtr)
if offset < 0 || offset > len(data) {
panic("subslice is not within data")
}
return offset
} }
// Raw returns the slice corresponding to the bytes in the given range. // Raw returns the slice corresponding to the bytes in the given range.
@@ -345,7 +356,7 @@ func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) {
b = p.parseWhitespace(b) b = p.parseWhitespace(b)
if len(b) == 0 { if len(b) == 0 {
return invalidReference, nil, NewParserError(b, "expected = after a key, but the document ends there") return invalidReference, nil, NewParserError(startB[:len(startB)-len(b)], "expected = after a key, but the document ends there")
} }
b, err = expect('=', b) b, err = expect('=', b)
@@ -363,9 +374,10 @@ func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) {
p.builder.Chain(valRef, key) p.builder.Chain(valRef, key)
p.builder.AttachChild(ref, valRef) p.builder.AttachChild(ref, valRef)
// Set Raw to span the entire key-value expression // Set Raw to span the entire key-value expression.
node := p.builder.NodeAt(ref) // Access the node directly in the slice to avoid the write barrier
node.Raw = p.rangeOfToken(startB[:len(startB)-len(b)], b) // that NodeAt's nodes-pointer setup would trigger.
p.builder.tree.nodes[ref].Raw = p.rangeOfToken(startB[:len(startB)-len(b)], b)
return ref, b, err return ref, b, err
} }
+36
View File
@@ -0,0 +1,36 @@
package unstable
import (
"errors"
"testing"
)
// Regression test for https://github.com/pelletier/go-toml/issues/1047:
// Parser.Range must use the real slice offset, not len(data)-len(slice).
func TestParser_Range_HighlightAfterComment(t *testing.T) {
input := []byte("# comment\n= \"value\"")
var p Parser
p.Reset(input)
for p.NextExpression() {
}
err := p.Error()
if err == nil {
t.Fatal("expected an error")
}
var perr *ParserError
if !errors.As(err, &perr) {
t.Fatalf("expected *ParserError, got %T", err)
}
r := p.Range(perr.Highlight)
shape := p.Shape(r)
if r.Offset != 10 {
t.Errorf("Range offset: got %d, want 10", r.Offset)
}
if shape.Start.Line != 2 || shape.Start.Column != 1 {
t.Errorf("position: got %d:%d, want 2:1", shape.Start.Line, shape.Start.Column)
}
}