Compare commits

..

7 Commits

Author SHA1 Message Date
Claude 8df3b65280 Preserve original formatting in Unmarshaler by using raw byte ranges
Instead of reconstructing key-value lines from parsed components, now
uses the original raw bytes from the document. This preserves:
- Whitespace around '=' (e.g., "key   =   value")
- String quoting style (basic vs literal)
- Number formats (hex, octal, binary)
- Inline table formatting

Changes:
- Add Raw range tracking to KeyValue expressions in parseKeyval
- Update handleKeyValuesUnmarshaler to use expr.Raw directly
- Remove keyNeedsQuoting helper (no longer needed)
- Add TestIssue873_FormattingPreservation test
- Update expected output in ExampleParser_comments
2026-01-26 21:30:16 +00:00
Claude 6c995ec13e Add Example tests and fix raw value extraction for boolean types
Add two godoc Example tests:
- ExampleDecoder_EnableUnmarshalerInterface_dynamicConfig: shows dynamic
  unmarshaling based on a type field
- ExampleDecoder_EnableUnmarshalerInterface_rawMessage: demonstrates
  RawMessage usage for deferred parsing

Fix handleKeyValuesUnmarshaler to handle values where Raw.Length == 0
(like boolean types) by using value.Data as fallback.
2026-01-26 21:30:15 +00:00
Claude 9f1bb6c97d Add double pointer test to achieve 100% coverage for handleKeyValues
Add TestIssue873_DoublePointerUnmarshaler to test pointer-to-pointer
to Unmarshaler types. This covers the pointer dereferencing loop in
handleKeyValues, bringing its coverage from 88% to 100%.

Total coverage: 97.4%
2026-01-26 21:30:15 +00:00
Claude d3b9283095 Add test for dotted keys to improve coverage
Add TestIssue873_DottedKeys to test dotted key handling (e.g., sub.key = value)
in the Unmarshaler interface. This improves coverage for handleKeyValuesUnmarshaler
from 93.3% to 96.7%.
2026-01-26 21:30:15 +00:00
Claude 013ffc24b8 Fix lint issues and improve test coverage for Unmarshaler interface
- Apply De Morgan's law in keyNeedsQuoting to satisfy staticcheck QF1001
- Remove unused splitTableUnmarshaler type from test
- Fix unused parameter lint warning in errorUnmarshaler873
- Add test for quoted keys that need special handling
- Add test for error propagation from UnmarshalTOML
- Update customTable873 parser to handle quoted keys properly

Coverage improved:
- handleKeyValuesUnmarshaler: 80.0% -> 93.3%
- keyNeedsQuoting: 66.7% -> 83.3%
- Overall main package: 97.2% -> 97.5%
2026-01-26 21:30:15 +00:00
Claude 2762e24a9c Implement bytes-based Unmarshaler interface for tables and arrays (#873)
This change brings back support for the unstable.Unmarshaler interface
for tables and array tables, addressing issue #873.

Key changes:
- Changed UnmarshalTOML signature from (*Node) to ([]byte) to provide
  raw TOML bytes instead of AST nodes
- Added RawMessage type (similar to json.RawMessage) for capturing raw
  TOML bytes for later processing
- Updated handleKeyValuesUnmarshaler to reconstruct key-value lines
  from the parsed keys and raw value bytes
- Added support for slice types implementing Unmarshaler (e.g., RawMessage)
- Removed unused AST helper functions from unstable/ast.go

The bytes-based interface allows users to:
- Get raw TOML bytes for custom parsing
- Delay TOML decoding using RawMessage
- Implement custom unmarshaling logic for complex types

Tests added for:
- Table unmarshaler with various scenarios
- Array table unmarshaler
- Split tables (same parent defined in multiple places)
- RawMessage usage
- Nested tables and mixed regular fields
2026-01-26 21:30:14 +00:00
Claude 5b6828661c Support Unmarshaler interface for tables and array tables (#873)
Extend the unstable.Unmarshaler interface support to work with tables
and array tables, not just single values.

When a type implementing unstable.Unmarshaler is the target of a table
(e.g., [table] or [[array]]), the UnmarshalTOML method receives a
synthetic InlineTable node containing all the key-value pairs belonging
to that table.

Key changes:
- Add handleKeyValuesUnmarshaler to collect and process table content
- Add copyExpressionNodes to deep-copy AST nodes for synthetic tables
- Add helper functions in unstable/ast.go for node manipulation
- Update documentation for EnableUnmarshalerInterface
- Add comprehensive tests for table and array table unmarshaling
2026-01-26 21:30:14 +00:00
16 changed files with 275 additions and 179 deletions
-25
View File
@@ -1,25 +0,0 @@
name: capabilities
on:
push:
branches:
- v2
pull_request:
branches:
- v2
jobs:
check:
name: check capabilities
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Setup go
uses: actions/setup-go@v6
with:
go-version: "1.26"
- name: Install capslock
run: go install github.com/google/capslock/cmd/capslock@latest
- name: Check for new capabilities
run: ./caps.sh check
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
dry-run: false
language: go
- name: Upload Crash
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v6
if: failure() && steps.build.outcome == 'success'
with:
name: artifacts
+1 -1
View File
@@ -15,6 +15,6 @@ jobs:
- name: Setup go
uses: actions/setup-go@v6
with:
go-version: "1.26"
go-version: "1.25"
- name: Run tests with coverage
run: ./ci.sh coverage -d "${GITHUB_BASE_REF-HEAD}"
+2 -2
View File
@@ -20,7 +20,7 @@ jobs:
fetch-depth: 0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
uses: docker/setup-buildx-action@v3
- name: Run Go versions compatibility test
run: |
@@ -28,7 +28,7 @@ jobs:
./test-go-versions.sh --output ./test-results $VERSIONS
- name: Upload test results
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v6
with:
name: go-versions-test-results
path: |
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
- name: Setup go
uses: actions/setup-go@v6
with:
go-version: "1.26"
go-version: "1.24"
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v9
with:
+3 -3
View File
@@ -22,15 +22,15 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version: "1.26"
go-version: "1.25"
- name: Login to GitHub Container Registry
uses: docker/login-action@v4
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v7
uses: goreleaser/goreleaser-action@v6
with:
distribution: goreleaser
version: '~> v2'
+1 -2
View File
@@ -10,10 +10,9 @@ on:
jobs:
build:
strategy:
fail-fast: false
matrix:
os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest', 'macos-14' ]
go: [ '1.25', '1.26' ]
go: [ '1.24', '1.25' ]
runs-on: ${{ matrix.os }}
name: ${{ matrix.go }}/${{ matrix.os }}
steps:
+3
View File
@@ -22,6 +22,7 @@ builds:
- linux_riscv64
- windows_amd64
- windows_arm64
- windows_arm
- darwin_amd64
- darwin_arm64
- id: tomljson
@@ -41,6 +42,7 @@ builds:
- linux_riscv64
- windows_amd64
- windows_arm64
- windows_arm
- darwin_amd64
- darwin_arm64
- id: jsontoml
@@ -60,6 +62,7 @@ builds:
- linux_arm
- windows_amd64
- windows_arm64
- windows_arm
- darwin_amd64
- darwin_arm64
universal_binaries:
+1 -10
View File
@@ -53,14 +53,6 @@ go-toml is a TOML library for Go. The goal is to provide an easy-to-use and effi
- Commit messages must explain **why** the change is needed
- Keep messages clear and informative even if details are in the PR description
### Capabilities
go-toml tracks system-level capabilities using [capslock](https://github.com/google/capslock). The baseline is in `capability_baseline.txt` and CI enforces that it does not grow.
- **Do not introduce new capabilities.** PRs that increase the capability set (e.g., adding network access, subprocess execution, syscalls) are unlikely to be accepted.
- If a change causes the capabilities check to fail, do not update the baseline to make it pass. Instead, rethink the approach to avoid requiring new capabilities.
- To check locally: `./caps.sh check` (requires `capslock` installed via `go install github.com/google/capslock/cmd/capslock@latest`)
## Pull Request Checklist
Before submitting:
@@ -69,5 +61,4 @@ Before submitting:
2. No backward-incompatible changes (unless discussed)
3. Relevant documentation added/updated
4. No performance regression (verify with benchmarks)
5. Capabilities are not increasing (`./caps.sh check`)
6. Title is clear and understandable for changelog
5. Title is clear and understandable for changelog
-19
View File
@@ -180,25 +180,6 @@ description. Pull requests that lower performance will receive more scrutiny.
[benchstat]: https://pkg.go.dev/golang.org/x/perf/cmd/benchstat
### Capabilities
We use [capslock](https://github.com/google/capslock) to track what
system-level capabilities (file access, network, syscalls, etc.) each package
requires. The current baseline is in `capability_baseline.txt`. CI will fail if
a change introduces a new capability.
**Pull requests that increase the set of capabilities are unlikely to be
accepted.** go-toml is a parsing library and should not need network access,
subprocess execution, or other capabilities beyond what it already uses.
If you believe a new capability is genuinely needed, discuss it in an issue
first. To update the baseline after approval:
```bash
go install github.com/google/capslock/cmd/capslock@latest
./caps.sh generate
```
### Style
Try to look around and follow the same format and structure as the rest of the
-1
View File
@@ -1 +0,0 @@
github.com/pelletier/go-toml/v2: CAPABILITY_REFLECT, CAPABILITY_UNANALYZED, CAPABILITY_UNSAFE_POINTER
-101
View File
@@ -1,101 +0,0 @@
#!/usr/bin/env bash
#
# Generates or checks the capability baseline for go-toml.
#
# Usage:
# ./caps.sh generate # regenerate capability_baseline.txt
# ./caps.sh check # check that capabilities haven't grown
#
# Requires: go, capslock (go install github.com/google/capslock/cmd/capslock@latest)
set -euo pipefail
BASELINE="capability_baseline.txt"
CAPSLOCK="${CAPSLOCK:-capslock}"
# Capabilities that must never appear in any package.
FORBIDDEN_CAPS=(
CAPABILITY_NETWORK
CAPABILITY_CGO
CAPABILITY_EXEC
)
capslock_to_baseline() {
"$CAPSLOCK" -packages=. -output=package -granularity=package \
| jq -r 'to_entries | sort_by(.key) | .[] | .key + ": " + (.value | sort | join(", "))'
}
generate() {
capslock_to_baseline > "$BASELINE"
echo "Wrote $BASELINE"
}
check() {
if [ ! -f "$BASELINE" ]; then
echo "ERROR: $BASELINE not found. Run '$0 generate' first."
exit 1
fi
current=$(mktemp)
trap 'rm -f "$current"' EXIT
capslock_to_baseline > "$current"
failed=0
# Check for forbidden capabilities in current output.
for cap in "${FORBIDDEN_CAPS[@]}"; do
if grep -q "$cap" "$current"; then
echo "FORBIDDEN capability found: $cap"
grep "$cap" "$current"
failed=1
fi
done
# Extract all unique capability names from baseline and current.
baseline_caps=$(grep -oE 'CAPABILITY_[A-Z_]+' "$BASELINE" | sort -u)
current_caps=$(grep -oE 'CAPABILITY_[A-Z_]+' "$current" | sort -u)
# Check for new capability names not in the baseline.
new_caps=$(comm -13 <(echo "$baseline_caps") <(echo "$current_caps"))
if [ -n "$new_caps" ]; then
echo "NEW capabilities detected (not in baseline):"
echo "$new_caps"
failed=1
fi
# Check for new per-package capabilities (a package gained a capability it didn't have before).
while IFS=': ' read -r pkg caps; do
baseline_pkg_caps=$(grep "^${pkg}:" "$BASELINE" 2>/dev/null | sed 's/^[^:]*: //' || true)
if [ -z "$baseline_pkg_caps" ]; then
echo "NEW package with capabilities: $pkg: $caps"
failed=1
continue
fi
# Check each capability in current for this package
for cap in $(echo "$caps" | tr ', ' '\n' | grep -v '^$'); do
if ! echo "$baseline_pkg_caps" | grep -q "$cap"; then
echo "NEW capability for $pkg: $cap"
failed=1
fi
done
done < "$current"
if [ "$failed" -eq 1 ]; then
echo ""
echo "FAILED: capabilities have grown."
echo "If this is intentional, run '$0 generate' and commit the updated $BASELINE."
exit 1
fi
echo "OK: no new capabilities detected."
}
case "${1:-}" in
generate) generate ;;
check) check ;;
*)
echo "Usage: $0 {generate|check}"
exit 1
;;
esac
-6
View File
@@ -259,12 +259,6 @@ func TestDecodeError_Position(t *testing.T) {
expectedRow: 3,
minCol: 5,
},
{
name: "missing equals on last line without trailing newline",
doc: "a = 1\nb = 2\nc",
expectedRow: 3,
minCol: 1,
},
}
for _, e := range examples {
+4 -5
View File
@@ -9,7 +9,7 @@ YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Go versions to test (1.11 through 1.26)
# Go versions to test (1.11 through 1.25)
GO_VERSIONS=(
"1.11"
"1.12"
@@ -26,7 +26,6 @@ GO_VERSIONS=(
"1.23"
"1.24"
"1.25"
"1.26"
)
# Default values
@@ -65,7 +64,7 @@ EXAMPLES:
$0 # Test all Go versions in parallel
$0 --sequential # Test all Go versions sequentially
$0 1.21 1.22 1.23 # Test specific versions
$0 --verbose --output ./results 1.25 1.26 # Verbose output to custom directory
$0 --verbose --output ./results 1.24 1.25 # Verbose output to custom directory
EXIT CODES:
0 Recent Go versions pass (good compatibility)
@@ -137,8 +136,8 @@ fi
# Validate Go versions
for version in "${GO_VERSIONS[@]}"; do
if ! [[ "$version" =~ ^1\.(1[1-9]|2[0-6])$ ]]; then
log_error "Invalid Go version: $version. Supported versions: 1.11-1.26"
if ! [[ "$version" =~ ^1\.(1[1-9]|2[0-5])$ ]]; then
log_error "Invalid Go version: $version. Supported versions: 1.11-1.25"
exit 1
fi
done
+257 -1
View File
@@ -4554,10 +4554,266 @@ func (c *customTable873) UnmarshalTOML(data []byte) error {
c.Keys = append(c.Keys, key)
c.Values[key] = string(valueBytes)
}
return nil
}
func TestIssue873_TableUnmarshaler(t *testing.T) {
type Config struct {
Section customTable873 `toml:"section"`
}
doc := `
[section]
key1 = "value1"
key2 = "value2"
key3 = "value3"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, []string{"key1", "key2", "key3"}, cfg.Section.Keys)
assert.Equal(t, "value1", cfg.Section.Values["key1"])
assert.Equal(t, "value2", cfg.Section.Values["key2"])
assert.Equal(t, "value3", cfg.Section.Values["key3"])
}
func TestIssue873_TableUnmarshaler_EmptyTable(t *testing.T) {
type Config struct {
Section customTable873 `toml:"section"`
}
doc := `
[section]
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, []string{}, cfg.Section.Keys)
}
func TestIssue873_ArrayTableUnmarshaler(t *testing.T) {
type Config struct {
Items []customTable873 `toml:"items"`
}
doc := `
[[items]]
name = "first"
id = "1"
[[items]]
name = "second"
id = "2"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, 2, len(cfg.Items))
assert.Equal(t, []string{"name", "id"}, cfg.Items[0].Keys)
assert.Equal(t, "first", cfg.Items[0].Values["name"])
assert.Equal(t, "1", cfg.Items[0].Values["id"])
assert.Equal(t, []string{"name", "id"}, cfg.Items[1].Keys)
assert.Equal(t, "second", cfg.Items[1].Values["name"])
assert.Equal(t, "2", cfg.Items[1].Values["id"])
}
func TestIssue873_NestedTableUnmarshaler(t *testing.T) {
type Config struct {
Outer struct {
Inner customTable873 `toml:"inner"`
} `toml:"outer"`
}
doc := `
[outer.inner]
a = "A"
b = "B"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, []string{"a", "b"}, cfg.Outer.Inner.Keys)
assert.Equal(t, "A", cfg.Outer.Inner.Values["a"])
assert.Equal(t, "B", cfg.Outer.Inner.Values["b"])
}
func TestIssue873_TableUnmarshaler_MultipleTables(t *testing.T) {
type Config struct {
First customTable873 `toml:"first"`
Second customTable873 `toml:"second"`
}
doc := `
[first]
key1 = "value1"
[second]
key2 = "value2"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, []string{"key1"}, cfg.First.Keys)
assert.Equal(t, "value1", cfg.First.Values["key1"])
assert.Equal(t, []string{"key2"}, cfg.Second.Keys)
assert.Equal(t, "value2", cfg.Second.Values["key2"])
}
// Test that regular struct fields still work alongside Unmarshaler tables
func TestIssue873_MixedWithRegularFields(t *testing.T) {
type Config struct {
Name string `toml:"name"`
Section customTable873 `toml:"section"`
Count int `toml:"count"`
}
doc := `
name = "test"
count = 42
[section]
foo = "bar"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, "test", cfg.Name)
assert.Equal(t, 42, cfg.Count)
assert.Equal(t, []string{"foo"}, cfg.Section.Keys)
assert.Equal(t, "bar", cfg.Section.Values["foo"])
}
// Test that pointer to Unmarshaler type works
func TestIssue873_PointerToUnmarshaler(t *testing.T) {
type Config struct {
Section *customTable873 `toml:"section"`
}
doc := `
[section]
hello = "world"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.True(t, cfg.Section != nil)
assert.Equal(t, []string{"hello"}, cfg.Section.Keys)
assert.Equal(t, "world", cfg.Section.Values["hello"])
}
// Test table with sub-tables defined separately
func TestIssue873_TableWithSubTables(t *testing.T) {
type Config struct {
Parent customTable873 `toml:"parent"`
}
doc := `
[parent]
name = "root"
[parent.child]
name = "nested"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
// The parent should only get the keys directly under [parent],
// not the [parent.child] sub-table
assert.NoError(t, err)
assert.Equal(t, []string{"name"}, cfg.Parent.Keys)
assert.Equal(t, "root", cfg.Parent.Values["name"])
}
// Test for issue #994 follow-up - tables defined piece-wise
// This addresses the maintainer's comment: "It doesn't deal with tables defined piece-wise yet"
func TestIssue994_TablesPieceWise(t *testing.T) {
// Test with piece-wise table definition (using [table] syntax)
// The customTable873 type captures key-value pairs in order,
// which is useful for use cases like maintaining map ordering
doc := `
[section]
first = "1"
second = "2"
third = "3"
`
type Config struct {
Section customTable873 `toml:"section"`
}
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
// Verify ordering is preserved (keys are collected in document order)
assert.Equal(t, []string{"first", "second", "third"}, cfg.Section.Keys)
assert.Equal(t, "1", cfg.Section.Values["first"])
assert.Equal(t, "2", cfg.Section.Values["second"])
assert.Equal(t, "3", cfg.Section.Values["third"])
}
// Test root-level struct with tables - combines #994 fix with #873 enhancement
func TestIssue994_RootWithTables(t *testing.T) {
type rootDoc struct {
Tables []customTable873 `toml:"tables"`
}
doc := `
[[tables]]
name = "first"
value = "one"
[[tables]]
name = "second"
value = "two"
`
var d rootDoc
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&d)
assert.NoError(t, err)
assert.Equal(t, 2, len(d.Tables))
assert.Equal(t, "first", d.Tables[0].Values["name"])
assert.Equal(t, "one", d.Tables[0].Values["value"])
assert.Equal(t, "second", d.Tables[1].Values["name"])
assert.Equal(t, "two", d.Tables[1].Values["value"])
}
// Test for split tables - when the same parent table is defined in multiple places
// This is a key requirement for issue #873: if type A implements Unmarshaler,
// and [a.b] and [a.d] are defined with another table [x] in between,
+1 -1
View File
@@ -345,7 +345,7 @@ func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) {
b = p.parseWhitespace(b)
if len(b) == 0 {
return invalidReference, nil, NewParserError(startB[:len(startB)-len(b)], "expected = after a key, but the document ends there")
return invalidReference, nil, NewParserError(b, "expected = after a key, but the document ends there")
}
b, err = expect('=', b)