Compare commits

..

11 Commits

Author SHA1 Message Date
Claude b7ffaf15eb Move regression tests to public Unmarshal API with full error string assertions
Remove the unstable package test and consolidate all test cases into
TestDecodeError_PositionAfterComment, which exercises toml.Unmarshal
and validates the complete human-readable error output including
context lines and tilde markers.

https://claude.ai/code/session_01EXYfFXc3DDGpQ27sWdXTKq
2026-04-12 11:59:46 +00:00
Claude 0248fc4c8c Fix incorrect error positions for non-suffix subslices in unstable.Parser.Range (#1047)
The unsafe removal (#1021) replaced danger.SubsliceOffset (pointer
arithmetic) with len(p.data)-len(b), which only works for suffix
slices. Parser.Range is called with arbitrary interior subslices
(e.g. ParserError.Highlight), so the offset was wrong whenever the
error occurred after previously scanned content like comments.

Fix by using reflect.ValueOf().Pointer() to recover the actual data
pointer, matching the approach already used in errors.go.

https://claude.ai/code/session_01EXYfFXc3DDGpQ27sWdXTKq
2026-04-12 11:52:16 +00:00
Thomas Pelletier f36a3ece9e Reduce marshal and unmarshal overhead (#1044)
* Reduce marshal and unmarshal overhead

Targeted optimizations to reduce performance overhead introduced by
recent feature additions and the unsafe removal.

Unmarshal:
- parseKeyval: access the node directly in the builder's slice to set
  Raw, bypassing NodeAt which triggers a GC write barrier for the
  nodes-pointer on every key-value expression.
- Iterator.Next: cache the *nodes slice dereference in a local variable
  to avoid repeated pointer-to-slice indirection in the hot loop.

Marshal:
- Guard shouldOmitZero calls with an inlineable options.omitzero check.
  shouldOmitZero has inlining cost 1145 (budget 80), so avoiding the
  function call when omitzero is not set removes per-field overhead.
- Inline the isNil check in encodeMap. isNil has inlining cost 93
  (budget 80), so expanding it at the single hot call site avoids
  per-map-entry function call overhead.

Update README benchmarks.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 11:08:39 +00:00
Thomas Pelletier 77f3862df4 Fix benchmark script replacing internal package imports (#1042)
* Fix benchmark script replacing internal package imports

The sed command in bench() was replacing all occurrences of the go-toml
module path, including sub-package imports like internal/assert. This
caused the BurntSushi/toml benchmark to fail because it tried to import
github.com/BurntSushi/toml/internal/assert which doesn't exist.

Fix by anchoring the sed pattern to only match the import path when
followed by a closing quote, preserving internal package imports.

Also add a guard in the benchstathtml Python script to give a clear
error instead of an IndexError when no benchmark results are available.

https://claude.ai/code/session_016JGASo49PeFSfCaDxvrGFE

* Update benchmark results in README

https://claude.ai/code/session_016JGASo49PeFSfCaDxvrGFE

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-23 22:00:18 -04:00
Thomas Pelletier 16b1ef5508 Fix parser error pointing to wrong line when last line has no trailing newline (#1041)
When parsing a key without '=' at EOF (e.g., "a = 1\nb = 2\nc"), the
error highlight was an empty slice, causing subsliceOffset to return 0
and the error to point at line 1 instead of line 3. Pass the consumed
key bytes as the highlight instead of the empty remainder.

Fixes #1032

https://claude.ai/code/session_01UWv8pyc8P1ktAPfHpveixj

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-23 21:34:12 -04:00
dependabot[bot] e14bde7c1d build(deps): bump docker/login-action from 3 to 4 (#1039)
Bumps [docker/login-action](https://github.com/docker/login-action) from 3 to 4.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 21:04:21 -04:00
dependabot[bot] 4b1ff01eb3 build(deps): bump docker/setup-buildx-action from 3 to 4 (#1040)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 21:04:11 -04:00
Thomas Pelletier 048a25f0f2 Go 1.26 (#1030)
* ci(release): drop unsupported windows/arm targets
2026-03-03 01:29:35 -05:00
dependabot[bot] b3575580f9 build(deps): bump goreleaser/goreleaser-action from 6 to 7 (#1035) 2026-03-03 00:47:47 -05:00
dependabot[bot] a0be52f4c1 build(deps): bump actions/upload-artifact from 6 to 7 (#1036) 2026-03-03 00:47:35 -05:00
Thomas Pelletier 316bfc66a4 Support Unmarshaler interface for tables and array tables (#1027)
Fixes #873

Extend the unstable.Unmarshaler interface support to work with tables
and array tables, not just single values.

When a type implementing unstable.Unmarshaler is the target of a table
(e.g., [table] or [[array]]), the UnmarshalTOML method receives a
synthetic InlineTable node containing all the key-value pairs belonging
to that table.

Key changes:
- Add handleKeyValuesUnmarshaler to collect and process table content
- Add copyExpressionNodes to deep-copy AST nodes for synthetic tables
- Add helper functions in unstable/ast.go for node manipulation
- Update documentation for EnableUnmarshalerInterface
- Add comprehensive tests for table and array table unmarshaling

* Implement bytes-based Unmarshaler interface for tables and arrays (#873)

This change brings back support for the unstable.Unmarshaler interface
for tables and array tables, addressing issue #873.

Key changes:
- Changed UnmarshalTOML signature from (*Node) to ([]byte) to provide
  raw TOML bytes instead of AST nodes
- Added RawMessage type (similar to json.RawMessage) for capturing raw
  TOML bytes for later processing
- Updated handleKeyValuesUnmarshaler to reconstruct key-value lines
  from the parsed keys and raw value bytes
- Added support for slice types implementing Unmarshaler (e.g., RawMessage)
- Removed unused AST helper functions from unstable/ast.go

The bytes-based interface allows users to:
- Get raw TOML bytes for custom parsing
- Delay TOML decoding using RawMessage
- Implement custom unmarshaling logic for complex types

Tests added for:
- Table unmarshaler with various scenarios
- Array table unmarshaler
- Split tables (same parent defined in multiple places)
- RawMessage usage
- Nested tables and mixed regular fields

* Fix lint issues and improve test coverage for Unmarshaler interface

- Apply De Morgan's law in keyNeedsQuoting to satisfy staticcheck QF1001
- Remove unused splitTableUnmarshaler type from test
- Fix unused parameter lint warning in errorUnmarshaler873
- Add test for quoted keys that need special handling
- Add test for error propagation from UnmarshalTOML
- Update customTable873 parser to handle quoted keys properly

Coverage improved:
- handleKeyValuesUnmarshaler: 80.0% -> 93.3%
- keyNeedsQuoting: 66.7% -> 83.3%
- Overall main package: 97.2% -> 97.5%

* Add test for dotted keys to improve coverage

Add TestIssue873_DottedKeys to test dotted key handling (e.g., sub.key = value)
in the Unmarshaler interface. This improves coverage for handleKeyValuesUnmarshaler
from 93.3% to 96.7%.

* Add double pointer test to achieve 100% coverage for handleKeyValues

Add TestIssue873_DoublePointerUnmarshaler to test pointer-to-pointer
to Unmarshaler types. This covers the pointer dereferencing loop in
handleKeyValues, bringing its coverage from 88% to 100%.

Total coverage: 97.4%

* Add Example tests and fix raw value extraction for boolean types

Add two godoc Example tests:
- ExampleDecoder_EnableUnmarshalerInterface_dynamicConfig: shows dynamic
  unmarshaling based on a type field
- ExampleDecoder_EnableUnmarshalerInterface_rawMessage: demonstrates
  RawMessage usage for deferred parsing

Fix handleKeyValuesUnmarshaler to handle values where Raw.Length == 0
(like boolean types) by using value.Data as fallback.

* Preserve original formatting in Unmarshaler by using raw byte ranges

Instead of reconstructing key-value lines from parsed components, now
uses the original raw bytes from the document. This preserves:
- Whitespace around '=' (e.g., "key   =   value")
- String quoting style (basic vs literal)
- Number formats (hex, octal, binary)
- Inline table formatting

Changes:
- Add Raw range tracking to KeyValue expressions in parseKeyval
- Update handleKeyValuesUnmarshaler to use expr.Raw directly
- Remove keyNeedsQuoting helper (no longer needed)
- Add TestIssue873_FormattingPreservation test
- Update expected output in ExampleParser_comments

* Prevent test matrix from canceling on first failure

Add fail-fast: false to the test workflow strategy so that all
OS/Go version combinations continue running even if one fails.
This provides better visibility into which specific combinations
have issues.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-11 09:57:23 -05:00
15 changed files with 154 additions and 320 deletions
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
dry-run: false
language: go
- name: Upload Crash
uses: actions/upload-artifact@v6
uses: actions/upload-artifact@v7
if: failure() && steps.build.outcome == 'success'
with:
name: artifacts
+1 -1
View File
@@ -15,6 +15,6 @@ jobs:
- name: Setup go
uses: actions/setup-go@v6
with:
go-version: "1.25"
go-version: "1.26"
- name: Run tests with coverage
run: ./ci.sh coverage -d "${GITHUB_BASE_REF-HEAD}"
+2 -2
View File
@@ -20,7 +20,7 @@ jobs:
fetch-depth: 0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Run Go versions compatibility test
run: |
@@ -28,7 +28,7 @@ jobs:
./test-go-versions.sh --output ./test-results $VERSIONS
- name: Upload test results
uses: actions/upload-artifact@v6
uses: actions/upload-artifact@v7
with:
name: go-versions-test-results
path: |
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
- name: Setup go
uses: actions/setup-go@v6
with:
go-version: "1.24"
go-version: "1.26"
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v9
with:
+3 -3
View File
@@ -22,15 +22,15 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version: "1.25"
go-version: "1.26"
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v6
uses: goreleaser/goreleaser-action@v7
with:
distribution: goreleaser
version: '~> v2'
+2 -1
View File
@@ -10,9 +10,10 @@ on:
jobs:
build:
strategy:
fail-fast: false
matrix:
os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest', 'macos-14' ]
go: [ '1.24', '1.25' ]
go: [ '1.25', '1.26' ]
runs-on: ${{ matrix.os }}
name: ${{ matrix.go }}/${{ matrix.os }}
steps:
-3
View File
@@ -22,7 +22,6 @@ builds:
- linux_riscv64
- windows_amd64
- windows_arm64
- windows_arm
- darwin_amd64
- darwin_arm64
- id: tomljson
@@ -42,7 +41,6 @@ builds:
- linux_riscv64
- windows_amd64
- windows_arm64
- windows_arm
- darwin_amd64
- darwin_arm64
- id: jsontoml
@@ -62,7 +60,6 @@ builds:
- linux_arm
- windows_amd64
- windows_arm64
- windows_arm
- darwin_amd64
- darwin_arm64
universal_binaries:
+17 -17
View File
@@ -239,12 +239,12 @@ Execution time speedup compared to other Go TOML libraries:
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead>
<tbody>
<tr><td>Marshal/HugoFrontMatter-2</td><td>1.9x</td><td>2.2x</td></tr>
<tr><td>Marshal/ReferenceFile/map-2</td><td>1.7x</td><td>2.1x</td></tr>
<tr><td>Marshal/ReferenceFile/struct-2</td><td>2.2x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/HugoFrontMatter-2</td><td>2.9x</td><td>2.7x</td></tr>
<tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.6x</td><td>2.7x</td></tr>
<tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.6x</td><td>5.1x</td></tr>
<tr><td>Marshal/HugoFrontMatter-2</td><td>2.1x</td><td>2.0x</td></tr>
<tr><td>Marshal/ReferenceFile/map-2</td><td>2.0x</td><td>2.0x</td></tr>
<tr><td>Marshal/ReferenceFile/struct-2</td><td>2.3x</td><td>2.5x</td></tr>
<tr><td>Unmarshal/HugoFrontMatter-2</td><td>3.3x</td><td>2.8x</td></tr>
<tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.9x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.8x</td><td>5.0x</td></tr>
</tbody>
</table>
<details><summary>See more</summary>
@@ -257,17 +257,17 @@ provided for completeness.</p>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead>
<tbody>
<tr><td>Marshal/SimpleDocument/map-2</td><td>1.8x</td><td>2.7x</td></tr>
<tr><td>Marshal/SimpleDocument/struct-2</td><td>2.7x</td><td>3.8x</td></tr>
<tr><td>Unmarshal/SimpleDocument/map-2</td><td>3.8x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/SimpleDocument/struct-2</td><td>5.6x</td><td>4.1x</td></tr>
<tr><td>UnmarshalDataset/example-2</td><td>3.0x</td><td>3.2x</td></tr>
<tr><td>UnmarshalDataset/code-2</td><td>2.3x</td><td>2.9x</td></tr>
<tr><td>UnmarshalDataset/twitter-2</td><td>2.6x</td><td>2.7x</td></tr>
<tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.2x</td><td>2.3x</td></tr>
<tr><td>UnmarshalDataset/canada-2</td><td>1.8x</td><td>1.5x</td></tr>
<tr><td>UnmarshalDataset/config-2</td><td>4.1x</td><td>2.9x</td></tr>
<tr><td>geomean</td><td>2.7x</td><td>2.8x</td></tr>
<tr><td>Marshal/SimpleDocument/map-2</td><td>2.0x</td><td>2.9x</td></tr>
<tr><td>Marshal/SimpleDocument/struct-2</td><td>2.5x</td><td>3.6x</td></tr>
<tr><td>Unmarshal/SimpleDocument/map-2</td><td>4.2x</td><td>3.4x</td></tr>
<tr><td>Unmarshal/SimpleDocument/struct-2</td><td>5.9x</td><td>4.4x</td></tr>
<tr><td>UnmarshalDataset/example-2</td><td>3.2x</td><td>2.9x</td></tr>
<tr><td>UnmarshalDataset/code-2</td><td>2.4x</td><td>2.8x</td></tr>
<tr><td>UnmarshalDataset/twitter-2</td><td>2.7x</td><td>2.5x</td></tr>
<tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.3x</td><td>2.3x</td></tr>
<tr><td>UnmarshalDataset/canada-2</td><td>1.9x</td><td>1.5x</td></tr>
<tr><td>UnmarshalDataset/config-2</td><td>5.4x</td><td>3.0x</td></tr>
<tr><td>geomean</td><td>2.9x</td><td>2.8x</td></tr>
</tbody>
</table>
<p>This table can be generated with <code>./ci.sh benchmark -a -html</code>.</p>
+6 -1
View File
@@ -147,7 +147,7 @@ bench() {
pushd "$dir"
if [ "${replace}" != "" ]; then
find ./benchmark/ -iname '*.go' -exec sed -i -E "s|github.com/pelletier/go-toml/v2|${replace}|g" {} \;
find ./benchmark/ -iname '*.go' -exec sed -i -E "s|github.com/pelletier/go-toml/v2\"|${replace}\"|g" {} \;
go get "${replace}"
fi
@@ -195,6 +195,11 @@ for line in reversed(lines[2:]):
"%.1fx" % (float(line[3])/v2), # v1
"%.1fx" % (float(line[7])/v2), # bs
])
if not results:
print("No benchmark results to display.", file=sys.stderr)
sys.exit(1)
# move geomean to the end
results.append(results[0])
del results[0]
+73
View File
@@ -259,6 +259,12 @@ func TestDecodeError_Position(t *testing.T) {
expectedRow: 3,
minCol: 5,
},
{
name: "missing equals on last line without trailing newline",
doc: "a = 1\nb = 2\nc",
expectedRow: 3,
minCol: 1,
},
}
for _, e := range examples {
@@ -295,6 +301,73 @@ OtherMissing = 1
assert.Equal(t, 2, len(strictErr.Unwrap()))
}
func TestDecodeError_PositionAfterComment(t *testing.T) {
// Regression test for https://github.com/pelletier/go-toml/issues/1047
// Error positions must be correct when the error occurs after comments or
// other content that was already scanned past.
examples := []struct {
desc string
doc string
expectedRow int
expectedCol int
expectedStr string
}{
{
desc: "invalid key after comment",
doc: "# comment\n= \"value\"",
expectedRow: 2,
expectedCol: 1,
expectedStr: "1| # comment\n2| = \"value\"\n | ~ invalid character at start of key: =",
},
{
desc: "invalid key after two comments",
doc: "# one\n# two\n= \"value\"",
expectedRow: 3,
expectedCol: 1,
expectedStr: "1| # one\n2| # two\n3| = \"value\"\n | ~ invalid character at start of key: =",
},
{
desc: "invalid key after key-value pair",
doc: "a = 1\n= 2",
expectedRow: 2,
expectedCol: 1,
expectedStr: "1| a = 1\n2| = 2\n | ~ invalid character at start of key: =",
},
{
desc: "invalid key after blank line",
doc: "a = 1\n\n= 2",
expectedRow: 3,
expectedCol: 1,
expectedStr: "1| a = 1\n2|\n3| = 2\n | ~ invalid character at start of key: =",
},
}
for _, e := range examples {
t.Run(e.desc, func(t *testing.T) {
var v interface{}
err := Unmarshal([]byte(e.doc), &v)
if err == nil {
t.Fatal("expected an error")
}
var derr *DecodeError
if !errors.As(err, &derr) {
t.Fatalf("error not a *DecodeError: %T: %v", err, err)
}
row, col := derr.Position()
if row != e.expectedRow {
t.Errorf("row: got %d, want %d (error: %s)", row, e.expectedRow, derr.String())
}
if col != e.expectedCol {
t.Errorf("col: got %d, want %d (error: %s)", col, e.expectedCol, derr.String())
}
assert.Equal(t, e.expectedStr, derr.String())
})
}
}
func ExampleDecodeError() {
doc := `name = 123__456`
+12 -9
View File
@@ -704,15 +704,18 @@ func (enc *Encoder) encodeMap(b []byte, ctx encoderCtx, v reflect.Value) ([]byte
for iter.Next() {
v := iter.Value()
if isNil(v) {
// For nil pointers, convert to zero value of the element type.
// This allows round-trip marshaling of maps with nil pointer values.
// For nil interfaces and nil maps, skip since we can't derive a type.
if v.Kind() == reflect.Ptr {
// Handle nil values: convert nil pointers to zero value,
// skip nil interfaces and nil maps.
switch v.Kind() {
case reflect.Ptr:
if v.IsNil() {
v = reflect.Zero(v.Type().Elem())
} else {
}
case reflect.Interface, reflect.Map:
if v.IsNil() {
continue
}
default:
}
k, err := enc.keyToString(iter.Key())
@@ -936,7 +939,7 @@ func (enc *Encoder) encodeTable(b []byte, ctx encoderCtx, t table) ([]byte, erro
if shouldOmitEmpty(kv.Options, kv.Value) {
continue
}
if shouldOmitZero(kv.Options, kv.Value) {
if kv.Options.omitzero && shouldOmitZero(kv.Options, kv.Value) {
continue
}
hasNonEmptyKV = true
@@ -958,7 +961,7 @@ func (enc *Encoder) encodeTable(b []byte, ctx encoderCtx, t table) ([]byte, erro
if shouldOmitEmpty(table.Options, table.Value) {
continue
}
if shouldOmitZero(table.Options, table.Value) {
if table.Options.omitzero && shouldOmitZero(table.Options, table.Value) {
continue
}
if first {
@@ -995,7 +998,7 @@ func (enc *Encoder) encodeTableInline(b []byte, ctx encoderCtx, t table) ([]byte
if shouldOmitEmpty(kv.Options, kv.Value) {
continue
}
if shouldOmitZero(kv.Options, kv.Value) {
if kv.Options.omitzero && shouldOmitZero(kv.Options, kv.Value) {
continue
}
+5 -4
View File
@@ -9,7 +9,7 @@ YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Go versions to test (1.11 through 1.25)
# Go versions to test (1.11 through 1.26)
GO_VERSIONS=(
"1.11"
"1.12"
@@ -26,6 +26,7 @@ GO_VERSIONS=(
"1.23"
"1.24"
"1.25"
"1.26"
)
# Default values
@@ -64,7 +65,7 @@ EXAMPLES:
$0 # Test all Go versions in parallel
$0 --sequential # Test all Go versions sequentially
$0 1.21 1.22 1.23 # Test specific versions
$0 --verbose --output ./results 1.24 1.25 # Verbose output to custom directory
$0 --verbose --output ./results 1.25 1.26 # Verbose output to custom directory
EXIT CODES:
0 Recent Go versions pass (good compatibility)
@@ -136,8 +137,8 @@ fi
# Validate Go versions
for version in "${GO_VERSIONS[@]}"; do
if ! [[ "$version" =~ ^1\.(1[1-9]|2[0-5])$ ]]; then
log_error "Invalid Go version: $version. Supported versions: 1.11-1.25"
if ! [[ "$version" =~ ^1\.(1[1-9]|2[0-6])$ ]]; then
log_error "Invalid Go version: $version. Supported versions: 1.11-1.26"
exit 1
fi
done
+1 -257
View File
@@ -4554,266 +4554,10 @@ func (c *customTable873) UnmarshalTOML(data []byte) error {
c.Keys = append(c.Keys, key)
c.Values[key] = string(valueBytes)
}
return nil
}
func TestIssue873_TableUnmarshaler(t *testing.T) {
type Config struct {
Section customTable873 `toml:"section"`
}
doc := `
[section]
key1 = "value1"
key2 = "value2"
key3 = "value3"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, []string{"key1", "key2", "key3"}, cfg.Section.Keys)
assert.Equal(t, "value1", cfg.Section.Values["key1"])
assert.Equal(t, "value2", cfg.Section.Values["key2"])
assert.Equal(t, "value3", cfg.Section.Values["key3"])
}
func TestIssue873_TableUnmarshaler_EmptyTable(t *testing.T) {
type Config struct {
Section customTable873 `toml:"section"`
}
doc := `
[section]
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, []string{}, cfg.Section.Keys)
}
func TestIssue873_ArrayTableUnmarshaler(t *testing.T) {
type Config struct {
Items []customTable873 `toml:"items"`
}
doc := `
[[items]]
name = "first"
id = "1"
[[items]]
name = "second"
id = "2"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, 2, len(cfg.Items))
assert.Equal(t, []string{"name", "id"}, cfg.Items[0].Keys)
assert.Equal(t, "first", cfg.Items[0].Values["name"])
assert.Equal(t, "1", cfg.Items[0].Values["id"])
assert.Equal(t, []string{"name", "id"}, cfg.Items[1].Keys)
assert.Equal(t, "second", cfg.Items[1].Values["name"])
assert.Equal(t, "2", cfg.Items[1].Values["id"])
}
func TestIssue873_NestedTableUnmarshaler(t *testing.T) {
type Config struct {
Outer struct {
Inner customTable873 `toml:"inner"`
} `toml:"outer"`
}
doc := `
[outer.inner]
a = "A"
b = "B"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, []string{"a", "b"}, cfg.Outer.Inner.Keys)
assert.Equal(t, "A", cfg.Outer.Inner.Values["a"])
assert.Equal(t, "B", cfg.Outer.Inner.Values["b"])
}
func TestIssue873_TableUnmarshaler_MultipleTables(t *testing.T) {
type Config struct {
First customTable873 `toml:"first"`
Second customTable873 `toml:"second"`
}
doc := `
[first]
key1 = "value1"
[second]
key2 = "value2"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, []string{"key1"}, cfg.First.Keys)
assert.Equal(t, "value1", cfg.First.Values["key1"])
assert.Equal(t, []string{"key2"}, cfg.Second.Keys)
assert.Equal(t, "value2", cfg.Second.Values["key2"])
}
// Test that regular struct fields still work alongside Unmarshaler tables
func TestIssue873_MixedWithRegularFields(t *testing.T) {
type Config struct {
Name string `toml:"name"`
Section customTable873 `toml:"section"`
Count int `toml:"count"`
}
doc := `
name = "test"
count = 42
[section]
foo = "bar"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, "test", cfg.Name)
assert.Equal(t, 42, cfg.Count)
assert.Equal(t, []string{"foo"}, cfg.Section.Keys)
assert.Equal(t, "bar", cfg.Section.Values["foo"])
}
// Test that pointer to Unmarshaler type works
func TestIssue873_PointerToUnmarshaler(t *testing.T) {
type Config struct {
Section *customTable873 `toml:"section"`
}
doc := `
[section]
hello = "world"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.True(t, cfg.Section != nil)
assert.Equal(t, []string{"hello"}, cfg.Section.Keys)
assert.Equal(t, "world", cfg.Section.Values["hello"])
}
// Test table with sub-tables defined separately
func TestIssue873_TableWithSubTables(t *testing.T) {
type Config struct {
Parent customTable873 `toml:"parent"`
}
doc := `
[parent]
name = "root"
[parent.child]
name = "nested"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
// The parent should only get the keys directly under [parent],
// not the [parent.child] sub-table
assert.NoError(t, err)
assert.Equal(t, []string{"name"}, cfg.Parent.Keys)
assert.Equal(t, "root", cfg.Parent.Values["name"])
}
// Test for issue #994 follow-up - tables defined piece-wise
// This addresses the maintainer's comment: "It doesn't deal with tables defined piece-wise yet"
func TestIssue994_TablesPieceWise(t *testing.T) {
// Test with piece-wise table definition (using [table] syntax)
// The customTable873 type captures key-value pairs in order,
// which is useful for use cases like maintaining map ordering
doc := `
[section]
first = "1"
second = "2"
third = "3"
`
type Config struct {
Section customTable873 `toml:"section"`
}
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
// Verify ordering is preserved (keys are collected in document order)
assert.Equal(t, []string{"first", "second", "third"}, cfg.Section.Keys)
assert.Equal(t, "1", cfg.Section.Values["first"])
assert.Equal(t, "2", cfg.Section.Values["second"])
assert.Equal(t, "3", cfg.Section.Values["third"])
}
// Test root-level struct with tables - combines #994 fix with #873 enhancement
func TestIssue994_RootWithTables(t *testing.T) {
type rootDoc struct {
Tables []customTable873 `toml:"tables"`
}
doc := `
[[tables]]
name = "first"
value = "one"
[[tables]]
name = "second"
value = "two"
`
var d rootDoc
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&d)
assert.NoError(t, err)
assert.Equal(t, 2, len(d.Tables))
assert.Equal(t, "first", d.Tables[0].Values["name"])
assert.Equal(t, "one", d.Tables[0].Values["value"])
assert.Equal(t, "second", d.Tables[1].Values["name"])
assert.Equal(t, "two", d.Tables[1].Values["value"])
}
// Test for split tables - when the same parent table is defined in multiple places
// This is a key requirement for issue #873: if type A implements Unmarshaler,
// and [a.b] and [a.d] are defined with another table [x] in between,
+7 -3
View File
@@ -28,12 +28,16 @@ func (c *Iterator) Next() bool {
if c.nodes == nil {
return false
}
nodes := *c.nodes
if !c.started {
c.started = true
} else if c.idx >= 0 {
c.idx = (*c.nodes)[c.idx].next
} else {
idx := c.idx
if idx >= 0 && int(idx) < len(nodes) {
c.idx = nodes[idx].next
}
return c.idx >= 0 && int(c.idx) < len(*c.nodes)
}
return c.idx >= 0 && int(c.idx) < len(nodes)
}
// IsLast returns true if the current node of the iterator is the last
+13 -7
View File
@@ -3,6 +3,7 @@ package unstable
import (
"bytes"
"fmt"
"reflect"
"unicode"
"github.com/pelletier/go-toml/v2/internal/characters"
@@ -83,10 +84,14 @@ func (p *Parser) rangeOfToken(token, rest []byte) Range {
}
// subsliceOffset returns the byte offset of subslice b within p.data.
// b must be a suffix (tail) of p.data.
// b must be a subslice of p.data (sharing the same backing array).
func (p *Parser) subsliceOffset(b []byte) int {
// b is a suffix of p.data, so its offset is len(p.data) - len(b)
return len(p.data) - len(b)
if len(b) == 0 {
return 0
}
dataPtr := reflect.ValueOf(p.data).Pointer()
subPtr := reflect.ValueOf(b).Pointer()
return int(subPtr - dataPtr)
}
// Raw returns the slice corresponding to the bytes in the given range.
@@ -345,7 +350,7 @@ func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) {
b = p.parseWhitespace(b)
if len(b) == 0 {
return invalidReference, nil, NewParserError(b, "expected = after a key, but the document ends there")
return invalidReference, nil, NewParserError(startB[:len(startB)-len(b)], "expected = after a key, but the document ends there")
}
b, err = expect('=', b)
@@ -363,9 +368,10 @@ func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) {
p.builder.Chain(valRef, key)
p.builder.AttachChild(ref, valRef)
// Set Raw to span the entire key-value expression
node := p.builder.NodeAt(ref)
node.Raw = p.rangeOfToken(startB[:len(startB)-len(b)], b)
// Set Raw to span the entire key-value expression.
// Access the node directly in the slice to avoid the write barrier
// that NodeAt's nodes-pointer setup would trigger.
p.builder.tree.nodes[ref].Raw = p.rangeOfToken(startB[:len(startB)-len(b)], b)
return ref, b, err
}