Compare commits

..

11 Commits

Author SHA1 Message Date
ostiwe a3d46d52a8 update module
test / release-check (push) Has been skipped
test / 1.25/macos-14 (push) Has been cancelled
test / 1.25/macos-latest (push) Has been cancelled
test / 1.25/ubuntu-latest (push) Has been cancelled
test / 1.25/windows-latest (push) Has been cancelled
test / 1.26/macos-14 (push) Has been cancelled
test / 1.26/macos-latest (push) Has been cancelled
test / 1.26/ubuntu-latest (push) Has been cancelled
test / 1.26/windows-latest (push) Has been cancelled
CodeQL / Analyze (go) (push) Has been cancelled
2026-04-14 12:40:39 +03:00
ostiwe d4c88c121f test
test / release-check (push) Has been skipped
CodeQL / Analyze (go) (push) Has been cancelled
test / 1.25/ubuntu-latest (push) Has been cancelled
test / 1.25/windows-latest (push) Has been cancelled
test / 1.26/macos-14 (push) Has been cancelled
test / 1.26/macos-latest (push) Has been cancelled
test / 1.26/ubuntu-latest (push) Has been cancelled
test / 1.26/windows-latest (push) Has been cancelled
test / 1.25/macos-latest (push) Has been cancelled
test / 1.25/macos-14 (push) Has been cancelled
2026-04-14 12:37:33 +03:00
Thomas Pelletier f36a3ece9e Reduce marshal and unmarshal overhead (#1044)
* Reduce marshal and unmarshal overhead

Targeted optimizations to reduce performance overhead introduced by
recent feature additions and the unsafe removal.

Unmarshal:
- parseKeyval: access the node directly in the builder's slice to set
  Raw, bypassing NodeAt which triggers a GC write barrier for the
  nodes-pointer on every key-value expression.
- Iterator.Next: cache the *nodes slice dereference in a local variable
  to avoid repeated pointer-to-slice indirection in the hot loop.

Marshal:
- Guard shouldOmitZero calls with an inlineable options.omitzero check.
  shouldOmitZero has inlining cost 1145 (budget 80), so avoiding the
  function call when omitzero is not set removes per-field overhead.
- Inline the isNil check in encodeMap. isNil has inlining cost 93
  (budget 80), so expanding it at the single hot call site avoids
  per-map-entry function call overhead.

Update README benchmarks.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 11:08:39 +00:00
Thomas Pelletier 77f3862df4 Fix benchmark script replacing internal package imports (#1042)
* Fix benchmark script replacing internal package imports

The sed command in bench() was replacing all occurrences of the go-toml
module path, including sub-package imports like internal/assert. This
caused the BurntSushi/toml benchmark to fail because it tried to import
github.com/BurntSushi/toml/internal/assert which doesn't exist.

Fix by anchoring the sed pattern to only match the import path when
followed by a closing quote, preserving internal package imports.

Also add a guard in the benchstathtml Python script to give a clear
error instead of an IndexError when no benchmark results are available.

https://claude.ai/code/session_016JGASo49PeFSfCaDxvrGFE

* Update benchmark results in README

https://claude.ai/code/session_016JGASo49PeFSfCaDxvrGFE

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-23 22:00:18 -04:00
Thomas Pelletier 16b1ef5508 Fix parser error pointing to wrong line when last line has no trailing newline (#1041)
When parsing a key without '=' at EOF (e.g., "a = 1\nb = 2\nc"), the
error highlight was an empty slice, causing subsliceOffset to return 0
and the error to point at line 1 instead of line 3. Pass the consumed
key bytes as the highlight instead of the empty remainder.

Fixes #1032

https://claude.ai/code/session_01UWv8pyc8P1ktAPfHpveixj

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-23 21:34:12 -04:00
dependabot[bot] e14bde7c1d build(deps): bump docker/login-action from 3 to 4 (#1039)
Bumps [docker/login-action](https://github.com/docker/login-action) from 3 to 4.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 21:04:21 -04:00
dependabot[bot] 4b1ff01eb3 build(deps): bump docker/setup-buildx-action from 3 to 4 (#1040)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 21:04:11 -04:00
Thomas Pelletier 048a25f0f2 Go 1.26 (#1030)
* ci(release): drop unsupported windows/arm targets
2026-03-03 01:29:35 -05:00
dependabot[bot] b3575580f9 build(deps): bump goreleaser/goreleaser-action from 6 to 7 (#1035) 2026-03-03 00:47:47 -05:00
dependabot[bot] a0be52f4c1 build(deps): bump actions/upload-artifact from 6 to 7 (#1036) 2026-03-03 00:47:35 -05:00
Thomas Pelletier 316bfc66a4 Support Unmarshaler interface for tables and array tables (#1027)
Fixes #873

Extend the unstable.Unmarshaler interface support to work with tables
and array tables, not just single values.

When a type implementing unstable.Unmarshaler is the target of a table
(e.g., [table] or [[array]]), the UnmarshalTOML method receives a
synthetic InlineTable node containing all the key-value pairs belonging
to that table.

Key changes:
- Add handleKeyValuesUnmarshaler to collect and process table content
- Add copyExpressionNodes to deep-copy AST nodes for synthetic tables
- Add helper functions in unstable/ast.go for node manipulation
- Update documentation for EnableUnmarshalerInterface
- Add comprehensive tests for table and array table unmarshaling

* Implement bytes-based Unmarshaler interface for tables and arrays (#873)

This change brings back support for the unstable.Unmarshaler interface
for tables and array tables, addressing issue #873.

Key changes:
- Changed UnmarshalTOML signature from (*Node) to ([]byte) to provide
  raw TOML bytes instead of AST nodes
- Added RawMessage type (similar to json.RawMessage) for capturing raw
  TOML bytes for later processing
- Updated handleKeyValuesUnmarshaler to reconstruct key-value lines
  from the parsed keys and raw value bytes
- Added support for slice types implementing Unmarshaler (e.g., RawMessage)
- Removed unused AST helper functions from unstable/ast.go

The bytes-based interface allows users to:
- Get raw TOML bytes for custom parsing
- Delay TOML decoding using RawMessage
- Implement custom unmarshaling logic for complex types

Tests added for:
- Table unmarshaler with various scenarios
- Array table unmarshaler
- Split tables (same parent defined in multiple places)
- RawMessage usage
- Nested tables and mixed regular fields

* Fix lint issues and improve test coverage for Unmarshaler interface

- Apply De Morgan's law in keyNeedsQuoting to satisfy staticcheck QF1001
- Remove unused splitTableUnmarshaler type from test
- Fix unused parameter lint warning in errorUnmarshaler873
- Add test for quoted keys that need special handling
- Add test for error propagation from UnmarshalTOML
- Update customTable873 parser to handle quoted keys properly

Coverage improved:
- handleKeyValuesUnmarshaler: 80.0% -> 93.3%
- keyNeedsQuoting: 66.7% -> 83.3%
- Overall main package: 97.2% -> 97.5%

* Add test for dotted keys to improve coverage

Add TestIssue873_DottedKeys to test dotted key handling (e.g., sub.key = value)
in the Unmarshaler interface. This improves coverage for handleKeyValuesUnmarshaler
from 93.3% to 96.7%.

* Add double pointer test to achieve 100% coverage for handleKeyValues

Add TestIssue873_DoublePointerUnmarshaler to test pointer-to-pointer
to Unmarshaler types. This covers the pointer dereferencing loop in
handleKeyValues, bringing its coverage from 88% to 100%.

Total coverage: 97.4%

* Add Example tests and fix raw value extraction for boolean types

Add two godoc Example tests:
- ExampleDecoder_EnableUnmarshalerInterface_dynamicConfig: shows dynamic
  unmarshaling based on a type field
- ExampleDecoder_EnableUnmarshalerInterface_rawMessage: demonstrates
  RawMessage usage for deferred parsing

Fix handleKeyValuesUnmarshaler to handle values where Raw.Length == 0
(like boolean types) by using value.Data as fallback.

* Preserve original formatting in Unmarshaler by using raw byte ranges

Instead of reconstructing key-value lines from parsed components, now
uses the original raw bytes from the document. This preserves:
- Whitespace around '=' (e.g., "key   =   value")
- String quoting style (basic vs literal)
- Number formats (hex, octal, binary)
- Inline table formatting

Changes:
- Add Raw range tracking to KeyValue expressions in parseKeyval
- Update handleKeyValuesUnmarshaler to use expr.Raw directly
- Remove keyNeedsQuoting helper (no longer needed)
- Add TestIssue873_FormattingPreservation test
- Update expected output in ExampleParser_comments

* Prevent test matrix from canceling on first failure

Add fail-fast: false to the test workflow strategy so that all
OS/Go version combinations continue running even if one fails.
This provides better visibility into which specific combinations
have issues.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-11 09:57:23 -05:00
55 changed files with 1309 additions and 2143 deletions
+1 -1
View File
@@ -5,7 +5,7 @@ Thank you for your pull request!
Please read the Code changes section of the CONTRIBUTING.md file, Please read the Code changes section of the CONTRIBUTING.md file,
and make sure you have followed the instructions. and make sure you have followed the instructions.
https://github.com/pelletier/go-toml/blob/v2/CONTRIBUTING.md#code-changes https://git.ostiwe.com/ostiwe/go-toml/blob/v2/CONTRIBUTING.md#code-changes
--> -->
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
dry-run: false dry-run: false
language: go language: go
- name: Upload Crash - name: Upload Crash
uses: actions/upload-artifact@v6 uses: actions/upload-artifact@v7
if: failure() && steps.build.outcome == 'success' if: failure() && steps.build.outcome == 'success'
with: with:
name: artifacts name: artifacts
+1 -1
View File
@@ -15,6 +15,6 @@ jobs:
- name: Setup go - name: Setup go
uses: actions/setup-go@v6 uses: actions/setup-go@v6
with: with:
go-version: "1.25" go-version: "1.26"
- name: Run tests with coverage - name: Run tests with coverage
run: ./ci.sh coverage -d "${GITHUB_BASE_REF-HEAD}" run: ./ci.sh coverage -d "${GITHUB_BASE_REF-HEAD}"
+2 -2
View File
@@ -20,7 +20,7 @@ jobs:
fetch-depth: 0 fetch-depth: 0
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v4
- name: Run Go versions compatibility test - name: Run Go versions compatibility test
run: | run: |
@@ -28,7 +28,7 @@ jobs:
./test-go-versions.sh --output ./test-results $VERSIONS ./test-go-versions.sh --output ./test-results $VERSIONS
- name: Upload test results - name: Upload test results
uses: actions/upload-artifact@v6 uses: actions/upload-artifact@v7
with: with:
name: go-versions-test-results name: go-versions-test-results
path: | path: |
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
- name: Setup go - name: Setup go
uses: actions/setup-go@v6 uses: actions/setup-go@v6
with: with:
go-version: "1.24" go-version: "1.26"
- name: Run golangci-lint - name: Run golangci-lint
uses: golangci/golangci-lint-action@v9 uses: golangci/golangci-lint-action@v9
with: with:
+3 -3
View File
@@ -22,15 +22,15 @@ jobs:
- name: Set up Go - name: Set up Go
uses: actions/setup-go@v6 uses: actions/setup-go@v6
with: with:
go-version: "1.25" go-version: "1.26"
- name: Login to GitHub Container Registry - name: Login to GitHub Container Registry
uses: docker/login-action@v3 uses: docker/login-action@v4
with: with:
registry: ghcr.io registry: ghcr.io
username: ${{ github.actor }} username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }} password: ${{ secrets.GITHUB_TOKEN }}
- name: Run GoReleaser - name: Run GoReleaser
uses: goreleaser/goreleaser-action@v6 uses: goreleaser/goreleaser-action@v7
with: with:
distribution: goreleaser distribution: goreleaser
version: '~> v2' version: '~> v2'
+2 -1
View File
@@ -10,9 +10,10 @@ on:
jobs: jobs:
build: build:
strategy: strategy:
fail-fast: false
matrix: matrix:
os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest', 'macos-14' ] os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest', 'macos-14' ]
go: [ '1.24', '1.25' ] go: [ '1.25', '1.26' ]
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
name: ${{ matrix.go }}/${{ matrix.os }} name: ${{ matrix.go }}/${{ matrix.os }}
steps: steps:
-3
View File
@@ -22,7 +22,6 @@ builds:
- linux_riscv64 - linux_riscv64
- windows_amd64 - windows_amd64
- windows_arm64 - windows_arm64
- windows_arm
- darwin_amd64 - darwin_amd64
- darwin_arm64 - darwin_arm64
- id: tomljson - id: tomljson
@@ -42,7 +41,6 @@ builds:
- linux_riscv64 - linux_riscv64
- windows_amd64 - windows_amd64
- windows_arm64 - windows_arm64
- windows_arm
- darwin_amd64 - darwin_amd64
- darwin_arm64 - darwin_arm64
- id: jsontoml - id: jsontoml
@@ -62,7 +60,6 @@ builds:
- linux_arm - linux_arm
- windows_amd64 - windows_amd64
- windows_arm64 - windows_arm64
- windows_arm
- darwin_amd64 - darwin_amd64
- darwin_arm64 - darwin_arm64
universal_binaries: universal_binaries:
+6 -6
View File
@@ -21,7 +21,7 @@ improvement, or new features that weren't envisioned before. Sometimes, a
seemingly innocent question leads to the fix of a bug. Don't hesitate and ask seemingly innocent question leads to the fix of a bug. Don't hesitate and ask
away! away!
[discussions]: https://github.com/pelletier/go-toml/discussions [discussions]: https://git.ostiwe.com/ostiwe/go-toml/discussions
## Improve the documentation ## Improve the documentation
@@ -224,12 +224,12 @@ Checklist:
5. If new version is an alpha or beta only, check pre-release box. 5. If new version is an alpha or beta only, check pre-release box.
[issues-tracker]: https://github.com/pelletier/go-toml/issues [issues-tracker]: https://git.ostiwe.com/ostiwe/go-toml/issues
[bug-report]: https://github.com/pelletier/go-toml/issues/new?template=bug_report.md [bug-report]: https://git.ostiwe.com/ostiwe/go-toml/issues/new?template=bug_report.md
[pkg.go.dev]: https://pkg.go.dev/github.com/pelletier/go-toml [pkg.go.dev]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml
[readme]: ./README.md [readme]: ./README.md
[fork]: https://help.github.com/articles/fork-a-repo [fork]: https://help.github.com/articles/fork-a-repo
[pull-request]: https://help.github.com/en/articles/creating-a-pull-request [pull-request]: https://help.github.com/en/articles/creating-a-pull-request
[new-release]: https://github.com/pelletier/go-toml/releases/new [new-release]: https://git.ostiwe.com/ostiwe/go-toml/releases/new
[gh]: https://github.com/cli/cli [gh]: https://github.com/cli/cli
[pr-labels]: https://github.com/pelletier/go-toml/blob/v2/.github/release.yml [pr-labels]: https://git.ostiwe.com/ostiwe/go-toml/blob/v2/.github/release.yml
+51 -51
View File
@@ -2,23 +2,23 @@
Go library for the [TOML](https://toml.io/en/) format. Go library for the [TOML](https://toml.io/en/) format.
This library supports [TOML v1.1.0](https://toml.io/en/v1.1.0). This library supports [TOML v1.0.0](https://toml.io/en/v1.0.0).
[🐞 Bug Reports](https://github.com/pelletier/go-toml/issues) [🐞 Bug Reports](https://git.ostiwe.com/ostiwe/go-toml/issues)
[💬 Anything else](https://github.com/pelletier/go-toml/discussions) [💬 Anything else](https://git.ostiwe.com/ostiwe/go-toml/discussions)
## Documentation ## Documentation
Full API, examples, and implementation notes are available in the Go Full API, examples, and implementation notes are available in the Go
documentation. documentation.
[![Go Reference](https://pkg.go.dev/badge/github.com/pelletier/go-toml/v2.svg)](https://pkg.go.dev/github.com/pelletier/go-toml/v2) [![Go Reference](https://pkg.go.dev/badge/git.ostiwe.com/ostiwe/go-toml/v2.svg)](https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2)
## Import ## Import
```go ```go
import "github.com/pelletier/go-toml/v2" import "git.ostiwe.com/ostiwe/go-toml/v2"
``` ```
See [Modules](#Modules). See [Modules](#Modules).
@@ -41,7 +41,7 @@ operations should not be shockingly slow. See [benchmarks](#benchmarks).
the TOML document was not present in the target structure. This is a great way the TOML document was not present in the target structure. This is a great way
to check for typos. [See example in the documentation][strict]. to check for typos. [See example in the documentation][strict].
[strict]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#example-Decoder.DisallowUnknownFields [strict]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#example-Decoder.DisallowUnknownFields
### Contextualized errors ### Contextualized errors
@@ -56,7 +56,7 @@ example:
3| port = 50 3| port = 50
``` ```
[decode-err]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#DecodeError [decode-err]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#DecodeError
### Local date and time support ### Local date and time support
@@ -67,10 +67,10 @@ this use-case, go-toml provides [`LocalDate`][tld], [`LocalTime`][tlt], and
making them convenient yet unambiguous structures for their respective TOML making them convenient yet unambiguous structures for their respective TOML
representation. representation.
[ldt]: https://toml.io/en/v1.1.0#local-date-time [ldt]: https://toml.io/en/v1.0.0#local-date-time
[tld]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#LocalDate [tld]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#LocalDate
[tlt]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#LocalTime [tlt]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#LocalTime
[tldt]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#LocalDateTime [tldt]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#LocalDateTime
### Commented config ### Commented config
@@ -90,7 +90,7 @@ port = 4242
# version = 'TLS 1.3' # version = 'TLS 1.3'
``` ```
[comments-example]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#example-Marshal-Commented [comments-example]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#example-Marshal-Commented
## Getting started ## Getting started
@@ -135,7 +135,7 @@ fmt.Println("tags:", cfg.Tags)
// tags: [go toml] // tags: [go toml]
``` ```
[unmarshal]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#Unmarshal [unmarshal]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#Unmarshal
Here is an example using tables with some simple nesting: Here is an example using tables with some simple nesting:
@@ -217,7 +217,7 @@ fmt.Println(string(b))
// Tags = ['go', 'toml'] // Tags = ['go', 'toml']
``` ```
[marshal]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#Marshal [marshal]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#Marshal
## Unstable API ## Unstable API
@@ -228,24 +228,24 @@ API subject to change.
### Parser ### Parser
Parser is the unstable API that allows iterative parsing of a TOML document at Parser is the unstable API that allows iterative parsing of a TOML document at
the AST level. See https://pkg.go.dev/github.com/pelletier/go-toml/v2/unstable. the AST level. See https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2/unstable.
## Benchmarks ## Benchmarks
Execution time speedup compared to other Go TOML libraries: Execution time speedup compared to other Go TOML libraries:
<table> <table>
<thead> <thead>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr> <tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead> </thead>
<tbody> <tbody>
<tr><td>Marshal/HugoFrontMatter-2</td><td>1.9x</td><td>2.2x</td></tr> <tr><td>Marshal/HugoFrontMatter-2</td><td>2.1x</td><td>2.0x</td></tr>
<tr><td>Marshal/ReferenceFile/map-2</td><td>1.7x</td><td>2.1x</td></tr> <tr><td>Marshal/ReferenceFile/map-2</td><td>2.0x</td><td>2.0x</td></tr>
<tr><td>Marshal/ReferenceFile/struct-2</td><td>2.2x</td><td>3.0x</td></tr> <tr><td>Marshal/ReferenceFile/struct-2</td><td>2.3x</td><td>2.5x</td></tr>
<tr><td>Unmarshal/HugoFrontMatter-2</td><td>2.9x</td><td>2.7x</td></tr> <tr><td>Unmarshal/HugoFrontMatter-2</td><td>3.3x</td><td>2.8x</td></tr>
<tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.6x</td><td>2.7x</td></tr> <tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.9x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.6x</td><td>5.1x</td></tr> <tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.8x</td><td>5.0x</td></tr>
</tbody> </tbody>
</table> </table>
<details><summary>See more</summary> <details><summary>See more</summary>
<p>The table above has the results of the most common use-cases. The table below <p>The table above has the results of the most common use-cases. The table below
@@ -253,22 +253,22 @@ contains the results of all benchmarks, including unrealistic ones. It is
provided for completeness.</p> provided for completeness.</p>
<table> <table>
<thead> <thead>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr> <tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead> </thead>
<tbody> <tbody>
<tr><td>Marshal/SimpleDocument/map-2</td><td>1.8x</td><td>2.7x</td></tr> <tr><td>Marshal/SimpleDocument/map-2</td><td>2.0x</td><td>2.9x</td></tr>
<tr><td>Marshal/SimpleDocument/struct-2</td><td>2.7x</td><td>3.8x</td></tr> <tr><td>Marshal/SimpleDocument/struct-2</td><td>2.5x</td><td>3.6x</td></tr>
<tr><td>Unmarshal/SimpleDocument/map-2</td><td>3.8x</td><td>3.0x</td></tr> <tr><td>Unmarshal/SimpleDocument/map-2</td><td>4.2x</td><td>3.4x</td></tr>
<tr><td>Unmarshal/SimpleDocument/struct-2</td><td>5.6x</td><td>4.1x</td></tr> <tr><td>Unmarshal/SimpleDocument/struct-2</td><td>5.9x</td><td>4.4x</td></tr>
<tr><td>UnmarshalDataset/example-2</td><td>3.0x</td><td>3.2x</td></tr> <tr><td>UnmarshalDataset/example-2</td><td>3.2x</td><td>2.9x</td></tr>
<tr><td>UnmarshalDataset/code-2</td><td>2.3x</td><td>2.9x</td></tr> <tr><td>UnmarshalDataset/code-2</td><td>2.4x</td><td>2.8x</td></tr>
<tr><td>UnmarshalDataset/twitter-2</td><td>2.6x</td><td>2.7x</td></tr> <tr><td>UnmarshalDataset/twitter-2</td><td>2.7x</td><td>2.5x</td></tr>
<tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.2x</td><td>2.3x</td></tr> <tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.3x</td><td>2.3x</td></tr>
<tr><td>UnmarshalDataset/canada-2</td><td>1.8x</td><td>1.5x</td></tr> <tr><td>UnmarshalDataset/canada-2</td><td>1.9x</td><td>1.5x</td></tr>
<tr><td>UnmarshalDataset/config-2</td><td>4.1x</td><td>2.9x</td></tr> <tr><td>UnmarshalDataset/config-2</td><td>5.4x</td><td>3.0x</td></tr>
<tr><td>geomean</td><td>2.7x</td><td>2.8x</td></tr> <tr><td>geomean</td><td>2.9x</td><td>2.8x</td></tr>
</tbody> </tbody>
</table> </table>
<p>This table can be generated with <code>./ci.sh benchmark -a -html</code>.</p> <p>This table can be generated with <code>./ci.sh benchmark -a -html</code>.</p>
</details> </details>
@@ -281,7 +281,7 @@ Installation instructions:
- Go ≥ 1.16: Nothing to do. Use the import in your code. The `go` command deals - Go ≥ 1.16: Nothing to do. Use the import in your code. The `go` command deals
with it automatically. with it automatically.
- Go ≥ 1.13: `GO111MODULE=on go get github.com/pelletier/go-toml/v2`. - Go ≥ 1.13: `GO111MODULE=on go get git.ostiwe.com/ostiwe/go-toml/v2`.
In case of trouble: [Go Modules FAQ][mod-faq]. In case of trouble: [Go Modules FAQ][mod-faq].
@@ -294,21 +294,21 @@ Go-toml provides three handy command line tools:
* `tomljson`: Reads a TOML file and outputs its JSON representation. * `tomljson`: Reads a TOML file and outputs its JSON representation.
``` ```
$ go install github.com/pelletier/go-toml/v2/cmd/tomljson@latest $ go install git.ostiwe.com/ostiwe/go-toml/v2/cmd/tomljson@latest
$ tomljson --help $ tomljson --help
``` ```
* `jsontoml`: Reads a JSON file and outputs a TOML representation. * `jsontoml`: Reads a JSON file and outputs a TOML representation.
``` ```
$ go install github.com/pelletier/go-toml/v2/cmd/jsontoml@latest $ go install git.ostiwe.com/ostiwe/go-toml/v2/cmd/jsontoml@latest
$ jsontoml --help $ jsontoml --help
``` ```
* `tomll`: Lints and reformats a TOML file. * `tomll`: Lints and reformats a TOML file.
``` ```
$ go install github.com/pelletier/go-toml/v2/cmd/tomll@latest $ go install git.ostiwe.com/ostiwe/go-toml/v2/cmd/tomll@latest
$ tomll --help $ tomll --help
``` ```
@@ -323,7 +323,7 @@ docker run -i ghcr.io/pelletier/go-toml:v2 tomljson < example.toml
Multiple versions are available on [ghcr.io][docker]. Multiple versions are available on [ghcr.io][docker].
[docker]: https://github.com/pelletier/go-toml/pkgs/container/go-toml [docker]: https://git.ostiwe.com/ostiwe/go-toml/pkgs/container/go-toml
## Migrating from v1 ## Migrating from v1
@@ -344,7 +344,7 @@ This could impact you if you are relying on casing to differentiate two fields,
and one of them is a not using the `toml` struct tag. The recommended solution and one of them is a not using the `toml` struct tag. The recommended solution
is to be specific about tag names for those fields using the `toml` struct tag. is to be specific about tag names for those fields using the `toml` struct tag.
[v1-keys]: https://github.com/pelletier/go-toml/blob/a2e52561804c6cd9392ebf0048ca64fe4af67a43/marshal.go#L775-L781 [v1-keys]: https://git.ostiwe.com/ostiwe/go-toml/blob/a2e52561804c6cd9392ebf0048ca64fe4af67a43/marshal.go#L775-L781
#### Ignore preexisting value in interface #### Ignore preexisting value in interface
@@ -544,7 +544,7 @@ fmt.Println("v2 Encoder:\n" + string(buf.Bytes()))
// key = 'value' // key = 'value'
``` ```
[sit]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#Encoder.SetIndentTables [sit]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#Encoder.SetIndentTables
#### Keys and strings are single quoted #### Keys and strings are single quoted
@@ -608,7 +608,7 @@ added to make the encoder behave correctly. Given backward compatibility is not
a problem anymore, v2 does the right thing by default: it follows the behavior a problem anymore, v2 does the right thing by default: it follows the behavior
of `encoding/json`. `Encoder.PromoteAnonymous` has been removed. of `encoding/json`. `Encoder.PromoteAnonymous` has been removed.
[nodoc]: https://github.com/pelletier/go-toml/discussions/506#discussioncomment-1526038 [nodoc]: https://git.ostiwe.com/ostiwe/go-toml/discussions/506#discussioncomment-1526038
### `query` ### `query`
@@ -620,7 +620,7 @@ This package has been removed because it was essentially not supported anymore
(last commit May 2020), increased the complexity of the code base, and more (last commit May 2020), increased the complexity of the code base, and more
complete solutions exist out there. complete solutions exist out there.
[query]: https://github.com/pelletier/go-toml/tree/f99d6bbca119636aeafcf351ee52b3d202782627/query [query]: https://git.ostiwe.com/ostiwe/go-toml/tree/f99d6bbca119636aeafcf351ee52b3d202782627/query
[dasel]: https://github.com/TomWright/dasel [dasel]: https://github.com/TomWright/dasel
## Versioning ## Versioning
+2 -2
View File
@@ -8,8 +8,8 @@ import (
"path/filepath" "path/filepath"
"testing" "testing"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
var benchInputs = []struct { var benchInputs = []struct {
+2 -2
View File
@@ -6,8 +6,8 @@ import (
"testing" "testing"
"time" "time"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
func TestUnmarshalSimple(t *testing.T) { func TestUnmarshalSimple(t *testing.T) {
+10 -5
View File
@@ -117,8 +117,8 @@ coverage() {
target_diff="${output_dir}/target.diff.txt" target_diff="${output_dir}/target.diff.txt"
head_diff="${output_dir}/head.diff.txt" head_diff="${output_dir}/head.diff.txt"
cat "${target_out}" | grep -E '^github.com/pelletier/go-toml' | tr -s "\t " | cut -f 2,3 | sort > "${target_diff}" cat "${target_out}" | grep -E '^git.ostiwe.com/ostiwe/go-toml' | tr -s "\t " | cut -f 2,3 | sort > "${target_diff}"
cat "${head_out}" | grep -E '^github.com/pelletier/go-toml' | tr -s "\t " | cut -f 2,3 | sort > "${head_diff}" cat "${head_out}" | grep -E '^git.ostiwe.com/ostiwe/go-toml' | tr -s "\t " | cut -f 2,3 | sort > "${head_diff}"
diff --side-by-side --suppress-common-lines "${target_diff}" "${head_diff}" diff --side-by-side --suppress-common-lines "${target_diff}" "${head_diff}"
return 1 return 1
@@ -147,7 +147,7 @@ bench() {
pushd "$dir" pushd "$dir"
if [ "${replace}" != "" ]; then if [ "${replace}" != "" ]; then
find ./benchmark/ -iname '*.go' -exec sed -i -E "s|github.com/pelletier/go-toml/v2|${replace}|g" {} \; find ./benchmark/ -iname '*.go' -exec sed -i -E "s|git.ostiwe.com/ostiwe/go-toml/v2\"|${replace}\"|g" {} \;
go get "${replace}" go get "${replace}"
fi fi
@@ -195,6 +195,11 @@ for line in reversed(lines[2:]):
"%.1fx" % (float(line[3])/v2), # v1 "%.1fx" % (float(line[3])/v2), # v1
"%.1fx" % (float(line[7])/v2), # bs "%.1fx" % (float(line[7])/v2), # bs
]) ])
if not results:
print("No benchmark results to display.", file=sys.stderr)
sys.exit(1)
# move geomean to the end # move geomean to the end
results.append(results[0]) results.append(results[0])
del results[0] del results[0]
@@ -252,9 +257,9 @@ benchmark() {
shift shift
v2stats=`fmktemp go-toml-v2` v2stats=`fmktemp go-toml-v2`
bench HEAD "${v2stats}" "github.com/pelletier/go-toml/v2" bench HEAD "${v2stats}" "git.ostiwe.com/ostiwe/go-toml/v2"
v1stats=`fmktemp go-toml-v1` v1stats=`fmktemp go-toml-v1`
bench HEAD "${v1stats}" "github.com/pelletier/go-toml" bench HEAD "${v1stats}" "git.ostiwe.com/ostiwe/go-toml"
bsstats=`fmktemp bs-toml` bsstats=`fmktemp bs-toml`
bench HEAD "${bsstats}" "github.com/BurntSushi/toml" bench HEAD "${bsstats}" "github.com/BurntSushi/toml"
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"os" "os"
"path" "path"
"github.com/pelletier/go-toml/v2/internal/testsuite" "git.ostiwe.com/ostiwe/go-toml/v2/internal/testsuite"
) )
func main() { func main() {
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"os" "os"
"path" "path"
"github.com/pelletier/go-toml/v2/internal/testsuite" "git.ostiwe.com/ostiwe/go-toml/v2/internal/testsuite"
) )
func main() { func main() {
+3 -3
View File
@@ -14,7 +14,7 @@
// //
// Using Go: // Using Go:
// //
// go install github.com/pelletier/go-toml/v2/cmd/jsontoml@latest // go install git.ostiwe.com/ostiwe/go-toml/v2/cmd/jsontoml@latest
package main package main
import ( import (
@@ -22,8 +22,8 @@ import (
"flag" "flag"
"io" "io"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/cli" "git.ostiwe.com/ostiwe/go-toml/v2/internal/cli"
) )
const usage = `jsontoml can be used in two ways: const usage = `jsontoml can be used in two ways:
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"strings" "strings"
"testing" "testing"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
func TestConvert(t *testing.T) { func TestConvert(t *testing.T) {
+3 -3
View File
@@ -14,7 +14,7 @@
// //
// Using Go: // Using Go:
// //
// go install github.com/pelletier/go-toml/v2/cmd/tomljson@latest // go install git.ostiwe.com/ostiwe/go-toml/v2/cmd/tomljson@latest
package main package main
import ( import (
@@ -23,8 +23,8 @@ import (
"fmt" "fmt"
"io" "io"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/cli" "git.ostiwe.com/ostiwe/go-toml/v2/internal/cli"
) )
const usage = `tomljson can be used in two ways: const usage = `tomljson can be used in two ways:
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"strings" "strings"
"testing" "testing"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
func TestConvert(t *testing.T) { func TestConvert(t *testing.T) {
+3 -3
View File
@@ -14,14 +14,14 @@
// //
// Using Go: // Using Go:
// //
// go install github.com/pelletier/go-toml/v2/cmd/tomll@latest // go install git.ostiwe.com/ostiwe/go-toml/v2/cmd/tomll@latest
package main package main
import ( import (
"io" "io"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/cli" "git.ostiwe.com/ostiwe/go-toml/v2/internal/cli"
) )
const usage = `tomll can be used in two ways: const usage = `tomll can be used in two ways:
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"strings" "strings"
"testing" "testing"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
func TestConvert(t *testing.T) { func TestConvert(t *testing.T) {
+1 -1
View File
@@ -3,7 +3,7 @@
// //
// Within the go-toml package, run `go generate`. Otherwise, use: // Within the go-toml package, run `go generate`. Otherwise, use:
// //
// go run github.com/pelletier/go-toml/cmd/tomltestgen -o toml_testgen_test.go // go run git.ostiwe.com/ostiwe/go-toml/cmd/tomltestgen -o toml_testgen_test.go
package main package main
import ( import (
+20 -25
View File
@@ -6,7 +6,7 @@ import (
"strconv" "strconv"
"time" "time"
"github.com/pelletier/go-toml/v2/unstable" "git.ostiwe.com/ostiwe/go-toml/v2/unstable"
) )
func parseInteger(b []byte) (int64, error) { func parseInteger(b []byte) (int64, error) {
@@ -162,7 +162,7 @@ func parseLocalDateTime(b []byte) (LocalDateTime, []byte, error) {
const localDateTimeByteMinLen = 11 const localDateTimeByteMinLen = 11
if len(b) < localDateTimeByteMinLen { if len(b) < localDateTimeByteMinLen {
return dt, nil, unstable.NewParserError(b, "local datetimes are expected to have the format YYYY-MM-DDTHH:MM[:SS[.NNNNNNNNN]]") return dt, nil, unstable.NewParserError(b, "local datetimes are expected to have the format YYYY-MM-DDTHH:MM:SS[.NNNNNNNNN]")
} }
date, err := parseLocalDate(b[:10]) date, err := parseLocalDate(b[:10])
@@ -194,10 +194,10 @@ func parseLocalTime(b []byte) (LocalTime, []byte, error) {
t LocalTime t LocalTime
) )
// check if b matches to have expected format HH:MM[:SS[.NNNNNN]] // check if b matches to have expected format HH:MM:SS[.NNNNNN]
const localTimeByteMinLen = 5 const localTimeByteLen = 8
if len(b) < localTimeByteMinLen { if len(b) < localTimeByteLen {
return t, nil, unstable.NewParserError(b, "times are expected to have the format HH:MM[:SS[.NNNNNN]]") return t, nil, unstable.NewParserError(b, "times are expected to have the format HH:MM:SS[.NNNNNN]")
} }
var err error var err error
@@ -221,26 +221,21 @@ func parseLocalTime(b []byte) (LocalTime, []byte, error) {
if t.Minute > 59 { if t.Minute > 59 {
return t, nil, unstable.NewParserError(b[3:5], "minutes cannot be greater 59") return t, nil, unstable.NewParserError(b[3:5], "minutes cannot be greater 59")
} }
if b[5] != ':' {
b = b[5:] return t, nil, unstable.NewParserError(b[5:6], "expecting colon between minutes and seconds")
if len(b) >= 1 && b[0] == ':' {
if len(b) < 3 {
return t, nil, unstable.NewParserError(b, "incomplete seconds")
}
t.Second, err = parseDecimalDigits(b[1:3])
if err != nil {
return t, nil, err
}
if t.Second > 59 {
return t, nil, unstable.NewParserError(b[1:3], "seconds cannot be greater than 59")
}
b = b[3:]
} }
t.Second, err = parseDecimalDigits(b[6:8])
if err != nil {
return t, nil, err
}
if t.Second > 59 {
return t, nil, unstable.NewParserError(b[6:8], "seconds cannot be greater than 59")
}
b = b[8:]
if len(b) >= 1 && b[0] == '.' { if len(b) >= 1 && b[0] == '.' {
frac := 0 frac := 0
precision := 0 precision := 0
@@ -263,7 +258,7 @@ func parseLocalTime(b []byte) (LocalTime, []byte, error) {
// to the supported precision and ignores the // to the supported precision and ignores the
// remaining digits. // remaining digits.
// //
// https://github.com/pelletier/go-toml/discussions/707 // https://git.ostiwe.com/ostiwe/go-toml/discussions/707
continue continue
} }
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"strconv" "strconv"
"strings" "strings"
"github.com/pelletier/go-toml/v2/unstable" "git.ostiwe.com/ostiwe/go-toml/v2/unstable"
) )
// DecodeError represents an error encountered during the parsing or decoding // DecodeError represents an error encountered during the parsing or decoding
+8 -2
View File
@@ -7,8 +7,8 @@ import (
"strings" "strings"
"testing" "testing"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
"github.com/pelletier/go-toml/v2/unstable" "git.ostiwe.com/ostiwe/go-toml/v2/unstable"
) )
//nolint:funlen //nolint:funlen
@@ -259,6 +259,12 @@ func TestDecodeError_Position(t *testing.T) {
expectedRow: 3, expectedRow: 3,
minCol: 5, minCol: 5,
}, },
{
name: "missing equals on last line without trailing newline",
doc: "a = 1\nb = 2\nc",
expectedRow: 3,
minCol: 1,
},
} }
for _, e := range examples { for _, e := range examples {
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"log" "log"
"strconv" "strconv"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
) )
type customInt int type customInt int
+2 -2
View File
@@ -3,8 +3,8 @@ package toml_test
import ( import (
"testing" "testing"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
func TestFastSimpleInt(t *testing.T) { func TestFastSimpleInt(t *testing.T) {
+2 -2
View File
@@ -5,8 +5,8 @@ import (
"strings" "strings"
"testing" "testing"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
func FuzzUnmarshal(f *testing.F) { func FuzzUnmarshal(f *testing.F) {
+1 -1
View File
@@ -1,3 +1,3 @@
module github.com/pelletier/go-toml/v2 module git.ostiwe.com/ostiwe/go-toml/v2
go 1.21.0 go 1.21.0
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"io" "io"
"os" "os"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
) )
type ConvertFn func(r io.Reader, w io.Writer) error type ConvertFn func(r io.Reader, w io.Writer) error
+2 -2
View File
@@ -9,8 +9,8 @@ import (
"strings" "strings"
"testing" "testing"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
func processMain(args []string, input io.Reader, stdout, stderr io.Writer, f ConvertFn) int { func processMain(args []string, input io.Reader, stdout, stderr io.Writer, f ConvertFn) int {
@@ -8,8 +8,8 @@ import (
"testing" "testing"
"time" "time"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
func TestDocMarshal(t *testing.T) { func TestDocMarshal(t *testing.T) {
@@ -15,8 +15,8 @@ import (
"testing" "testing"
"time" "time"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
type basicMarshalTestStruct struct { type basicMarshalTestStruct struct {
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"strconv" "strconv"
"time" "time"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
) )
// addTag adds JSON tags to a data structure as expected by toml-test. // addTag adds JSON tags to a data structure as expected by toml-test.
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"strconv" "strconv"
"time" "time"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
) )
// Remove JSON tags to a data structure as returned by toml-test. // Remove JSON tags to a data structure as returned by toml-test.
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"fmt" "fmt"
"os" "os"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
) )
// Marshal is a helper function for calling toml.Marshal // Marshal is a helper function for calling toml.Marshal
+1 -1
View File
@@ -1,6 +1,6 @@
package tracker package tracker
import "github.com/pelletier/go-toml/v2/unstable" import "git.ostiwe.com/ostiwe/go-toml/v2/unstable"
// KeyTracker is a tracker that keeps track of the current Key as the AST is // KeyTracker is a tracker that keeps track of the current Key as the AST is
// walked. // walked.
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"fmt" "fmt"
"sync" "sync"
"github.com/pelletier/go-toml/v2/unstable" "git.ostiwe.com/ostiwe/go-toml/v2/unstable"
) )
type keyKind uint8 type keyKind uint8
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"reflect" "reflect"
"testing" "testing"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
func TestEntrySize(t *testing.T) { func TestEntrySize(t *testing.T) {
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"strings" "strings"
"time" "time"
"github.com/pelletier/go-toml/v2/unstable" "git.ostiwe.com/ostiwe/go-toml/v2/unstable"
) )
// LocalDate represents a calendar day in no specific timezone. // LocalDate represents a calendar day in no specific timezone.
+2 -9
View File
@@ -4,8 +4,8 @@ import (
"testing" "testing"
"time" "time"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
func TestLocalDate_AsTime(t *testing.T) { func TestLocalDate_AsTime(t *testing.T) {
@@ -67,13 +67,6 @@ func TestLocalTime_UnmarshalMarshalText(t *testing.T) {
assert.Error(t, err) assert.Error(t, err)
} }
func TestLocalTime_UnmarshalText_WithoutSeconds(t *testing.T) {
d := toml.LocalTime{}
err := d.UnmarshalText([]byte("14:15"))
assert.NoError(t, err)
assert.Equal(t, toml.LocalTime{14, 15, 0, 0, 0}, d)
}
func TestLocalTime_RoundTrip(t *testing.T) { func TestLocalTime_RoundTrip(t *testing.T) {
var d struct{ A toml.LocalTime } var d struct{ A toml.LocalTime }
err := toml.Unmarshal([]byte("a=20:12:01.500"), &d) err := toml.Unmarshal([]byte("a=20:12:01.500"), &d)
+16 -13
View File
@@ -15,7 +15,7 @@ import (
"time" "time"
"unicode" "unicode"
"github.com/pelletier/go-toml/v2/internal/characters" "git.ostiwe.com/ostiwe/go-toml/v2/internal/characters"
) )
// Marshal serializes a Go value as a TOML document. // Marshal serializes a Go value as a TOML document.
@@ -271,7 +271,7 @@ func (enc *Encoder) encode(b []byte, ctx encoderCtx, v reflect.Value) ([]byte, e
return append(b, x.String()...), nil return append(b, x.String()...), nil
case json.Number: case json.Number:
if enc.marshalJSONNumbers { if enc.marshalJSONNumbers {
if x == "" { /// Useful zero value. if x == "" { // / Useful zero value.
return append(b, "0"...), nil return append(b, "0"...), nil
} else if v, err := x.Int64(); err == nil { } else if v, err := x.Int64(); err == nil {
return enc.encode(b, ctx, reflect.ValueOf(v)) return enc.encode(b, ctx, reflect.ValueOf(v))
@@ -704,15 +704,18 @@ func (enc *Encoder) encodeMap(b []byte, ctx encoderCtx, v reflect.Value) ([]byte
for iter.Next() { for iter.Next() {
v := iter.Value() v := iter.Value()
if isNil(v) { // Handle nil values: convert nil pointers to zero value,
// For nil pointers, convert to zero value of the element type. // skip nil interfaces and nil maps.
// This allows round-trip marshaling of maps with nil pointer values. switch v.Kind() {
// For nil interfaces and nil maps, skip since we can't derive a type. case reflect.Ptr:
if v.Kind() == reflect.Ptr { if v.IsNil() {
v = reflect.Zero(v.Type().Elem()) v = reflect.Zero(v.Type().Elem())
} else { }
case reflect.Interface, reflect.Map:
if v.IsNil() {
continue continue
} }
default:
} }
k, err := enc.keyToString(iter.Key()) k, err := enc.keyToString(iter.Key())
@@ -936,7 +939,7 @@ func (enc *Encoder) encodeTable(b []byte, ctx encoderCtx, t table) ([]byte, erro
if shouldOmitEmpty(kv.Options, kv.Value) { if shouldOmitEmpty(kv.Options, kv.Value) {
continue continue
} }
if shouldOmitZero(kv.Options, kv.Value) { if kv.Options.omitzero && shouldOmitZero(kv.Options, kv.Value) {
continue continue
} }
hasNonEmptyKV = true hasNonEmptyKV = true
@@ -958,7 +961,7 @@ func (enc *Encoder) encodeTable(b []byte, ctx encoderCtx, t table) ([]byte, erro
if shouldOmitEmpty(table.Options, table.Value) { if shouldOmitEmpty(table.Options, table.Value) {
continue continue
} }
if shouldOmitZero(table.Options, table.Value) { if table.Options.omitzero && shouldOmitZero(table.Options, table.Value) {
continue continue
} }
if first { if first {
@@ -995,7 +998,7 @@ func (enc *Encoder) encodeTableInline(b []byte, ctx encoderCtx, t table) ([]byte
if shouldOmitEmpty(kv.Options, kv.Value) { if shouldOmitEmpty(kv.Options, kv.Value) {
continue continue
} }
if shouldOmitZero(kv.Options, kv.Value) { if kv.Options.omitzero && shouldOmitZero(kv.Options, kv.Value) {
continue continue
} }
@@ -1107,7 +1110,7 @@ func (enc *Encoder) encodeSliceAsArrayTable(b []byte, ctx encoderCtx, v reflect.
scratch = enc.indent(ctx.indent, scratch) scratch = enc.indent(ctx.indent, scratch)
} }
scratch = append(scratch, "[["...) scratch = append(scratch, "["...)
for i, k := range ctx.parentKey { for i, k := range ctx.parentKey {
if i > 0 { if i > 0 {
@@ -1117,7 +1120,7 @@ func (enc *Encoder) encodeSliceAsArrayTable(b []byte, ctx encoderCtx, v reflect.
scratch = enc.encodeKey(scratch, k) scratch = enc.encodeKey(scratch, k)
} }
scratch = append(scratch, "]]\n"...) scratch = append(scratch, "]\n"...)
ctx.skipTableHeader = true ctx.skipTableHeader = true
b = enc.encodeComment(ctx.indent, ctx.options.comment, b) b = enc.encodeComment(ctx.indent, ctx.options.comment, b)
+3 -3
View File
@@ -13,8 +13,8 @@ import (
"testing" "testing"
"time" "time"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
type marshalTextKey struct { type marshalTextKey struct {
@@ -2220,7 +2220,7 @@ port = 4242
// TestMarshalIssue975 tests that nil pointer values in maps are marshaled as // TestMarshalIssue975 tests that nil pointer values in maps are marshaled as
// empty tables, allowing round-trip marshaling to work correctly. // empty tables, allowing round-trip marshaling to work correctly.
// See https://github.com/pelletier/go-toml/issues/975 // See https://git.ostiwe.com/ostiwe/go-toml/issues/975
func TestMarshalIssue975(t *testing.T) { func TestMarshalIssue975(t *testing.T) {
// Test case from the issue: map[string]*struct{} // Test case from the issue: map[string]*struct{}
oldMap := map[string]*struct{}{ oldMap := map[string]*struct{}{
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"reflect" "reflect"
"strings" "strings"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
) )
// FuzzToml is the fuzzing target. // FuzzToml is the fuzzing target.
+2 -2
View File
@@ -1,8 +1,8 @@
package toml package toml
import ( import (
"github.com/pelletier/go-toml/v2/internal/tracker" "git.ostiwe.com/ostiwe/go-toml/v2/internal/tracker"
"github.com/pelletier/go-toml/v2/unstable" "git.ostiwe.com/ostiwe/go-toml/v2/unstable"
) )
type strict struct { type strict struct {
+5 -4
View File
@@ -9,7 +9,7 @@ YELLOW='\033[1;33m'
BLUE='\033[0;34m' BLUE='\033[0;34m'
NC='\033[0m' # No Color NC='\033[0m' # No Color
# Go versions to test (1.11 through 1.25) # Go versions to test (1.11 through 1.26)
GO_VERSIONS=( GO_VERSIONS=(
"1.11" "1.11"
"1.12" "1.12"
@@ -26,6 +26,7 @@ GO_VERSIONS=(
"1.23" "1.23"
"1.24" "1.24"
"1.25" "1.25"
"1.26"
) )
# Default values # Default values
@@ -64,7 +65,7 @@ EXAMPLES:
$0 # Test all Go versions in parallel $0 # Test all Go versions in parallel
$0 --sequential # Test all Go versions sequentially $0 --sequential # Test all Go versions sequentially
$0 1.21 1.22 1.23 # Test specific versions $0 1.21 1.22 1.23 # Test specific versions
$0 --verbose --output ./results 1.24 1.25 # Verbose output to custom directory $0 --verbose --output ./results 1.25 1.26 # Verbose output to custom directory
EXIT CODES: EXIT CODES:
0 Recent Go versions pass (good compatibility) 0 Recent Go versions pass (good compatibility)
@@ -136,8 +137,8 @@ fi
# Validate Go versions # Validate Go versions
for version in "${GO_VERSIONS[@]}"; do for version in "${GO_VERSIONS[@]}"; do
if ! [[ "$version" =~ ^1\.(1[1-9]|2[0-5])$ ]]; then if ! [[ "$version" =~ ^1\.(1[1-9]|2[0-6])$ ]]; then
log_error "Invalid Go version: $version. Supported versions: 1.11-1.25" log_error "Invalid Go version: $version. Supported versions: 1.11-1.26"
exit 1 exit 1
fi fi
done done
+5 -5
View File
@@ -1,5 +1,5 @@
//go:generate go run github.com/toml-lang/toml-test/v2/cmd/toml-test@v2.1.0 copy -toml 1.1 ./tests //go:generate go run github.com/toml-lang/toml-test/cmd/toml-test@v1.6.0 -copy ./tests
//go:generate go run ./cmd/tomltestgen/main.go -r v2.1.0 -o toml_testgen_test.go //go:generate go run ./cmd/tomltestgen/main.go -r v1.6.0 -o toml_testgen_test.go
package toml_test package toml_test
@@ -8,9 +8,9 @@ import (
"errors" "errors"
"testing" "testing"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
"github.com/pelletier/go-toml/v2/internal/testsuite" "git.ostiwe.com/ostiwe/go-toml/v2/internal/testsuite"
) )
func testgenInvalid(t *testing.T, input string) { func testgenInvalid(t *testing.T, input string) {
+615 -1234
View File
File diff suppressed because it is too large Load Diff
+87 -17
View File
@@ -12,8 +12,8 @@ import (
"sync/atomic" "sync/atomic"
"time" "time"
"github.com/pelletier/go-toml/v2/internal/tracker" "git.ostiwe.com/ostiwe/go-toml/v2/internal/tracker"
"github.com/pelletier/go-toml/v2/unstable" "git.ostiwe.com/ostiwe/go-toml/v2/unstable"
) )
// Unmarshal deserializes a TOML document into a Go value. // Unmarshal deserializes a TOML document into a Go value.
@@ -56,13 +56,18 @@ func (d *Decoder) DisallowUnknownFields() *Decoder {
// EnableUnmarshalerInterface allows to enable unmarshaler interface. // EnableUnmarshalerInterface allows to enable unmarshaler interface.
// //
// With this feature enabled, types implementing the unstable/Unmarshaler // With this feature enabled, types implementing the unstable.Unmarshaler
// interface can be decoded from any structure of the document. It allows types // interface can be decoded from any structure of the document. It allows types
// that don't have a straightforward TOML representation to provide their own // that don't have a straightforward TOML representation to provide their own
// decoding logic. // decoding logic.
// //
// Currently, types can only decode from a single value. Tables and array tables // The UnmarshalTOML method receives raw TOML bytes:
// are not supported. // - For single values: the raw value bytes (e.g., `"hello"` for a string)
// - For tables: all key-value lines belonging to that table
// - For inline tables/arrays: the raw bytes of the inline structure
//
// The unstable.RawMessage type can be used to capture raw TOML bytes for
// later processing, similar to json.RawMessage.
// //
// *Unstable:* This method does not follow the compatibility guarantees of // *Unstable:* This method does not follow the compatibility guarantees of
// semver. It can be changed or removed without a new major version being // semver. It can be changed or removed without a new major version being
@@ -599,18 +604,28 @@ func (d *decoder) handleArrayTablePart(key unstable.Iterator, v reflect.Value) (
// cannot handle it. // cannot handle it.
func (d *decoder) handleTable(key unstable.Iterator, v reflect.Value) (reflect.Value, error) { func (d *decoder) handleTable(key unstable.Iterator, v reflect.Value) (reflect.Value, error) {
if v.Kind() == reflect.Slice { if v.Kind() == reflect.Slice {
if v.Len() == 0 { // For non-empty slices, work with the last element
return reflect.Value{}, unstable.NewParserError(key.Node().Data, "cannot store a table in a slice") if v.Len() > 0 {
elem := v.Index(v.Len() - 1)
x, err := d.handleTable(key, elem)
if err != nil {
return reflect.Value{}, err
}
if x.IsValid() {
elem.Set(x)
}
return reflect.Value{}, nil
} }
elem := v.Index(v.Len() - 1) // Empty slice - check if it implements Unmarshaler (e.g., RawMessage)
x, err := d.handleTable(key, elem) // and we're at the end of the key path
if err != nil { if d.unmarshalerInterface && !key.Next() {
return reflect.Value{}, err if v.CanAddr() && v.Addr().CanInterface() {
if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok {
return d.handleKeyValuesUnmarshaler(outi)
}
}
} }
if x.IsValid() { return reflect.Value{}, unstable.NewParserError(key.Node().Data, "cannot store a table in a slice")
elem.Set(x)
}
return reflect.Value{}, nil
} }
if key.Next() { if key.Next() {
// Still scoping the key // Still scoping the key
@@ -624,6 +639,24 @@ func (d *decoder) handleTable(key unstable.Iterator, v reflect.Value) (reflect.V
// Handle root expressions until the end of the document or the next // Handle root expressions until the end of the document or the next
// non-key-value. // non-key-value.
func (d *decoder) handleKeyValues(v reflect.Value) (reflect.Value, error) { func (d *decoder) handleKeyValues(v reflect.Value) (reflect.Value, error) {
// Check if target implements Unmarshaler before processing key-values.
// This allows types to handle entire tables themselves.
if d.unmarshalerInterface {
vv := v
for vv.Kind() == reflect.Ptr {
if vv.IsNil() {
vv.Set(reflect.New(vv.Type().Elem()))
}
vv = vv.Elem()
}
if vv.CanAddr() && vv.Addr().CanInterface() {
if outi, ok := vv.Addr().Interface().(unstable.Unmarshaler); ok {
// Collect all key-value expressions for this table
return d.handleKeyValuesUnmarshaler(outi)
}
}
}
var rv reflect.Value var rv reflect.Value
for d.nextExpr() { for d.nextExpr() {
expr := d.expr() expr := d.expr()
@@ -653,6 +686,41 @@ func (d *decoder) handleKeyValues(v reflect.Value) (reflect.Value, error) {
return rv, nil return rv, nil
} }
// handleKeyValuesUnmarshaler collects all key-value expressions for a table
// and passes them to the Unmarshaler as raw TOML bytes.
func (d *decoder) handleKeyValuesUnmarshaler(u unstable.Unmarshaler) (reflect.Value, error) {
// Collect raw bytes from all key-value expressions for this table.
// We use the Raw field on each KeyValue expression to preserve the
// original formatting (whitespace, quoting style, etc.) from the document.
var buf []byte
for d.nextExpr() {
expr := d.expr()
if expr.Kind != unstable.KeyValue {
d.stashExpr()
break
}
_, err := d.seen.CheckExpression(expr)
if err != nil {
return reflect.Value{}, err
}
// Use the raw bytes from the original document to preserve formatting
if expr.Raw.Length > 0 {
raw := d.p.Raw(expr.Raw)
buf = append(buf, raw...)
}
buf = append(buf, '\n')
}
if err := u.UnmarshalTOML(buf); err != nil {
return reflect.Value{}, err
}
return reflect.Value{}, nil
}
type ( type (
handlerFn func(key unstable.Iterator, v reflect.Value) (reflect.Value, error) handlerFn func(key unstable.Iterator, v reflect.Value) (reflect.Value, error)
valueMakerFn func() reflect.Value valueMakerFn func() reflect.Value
@@ -697,7 +765,8 @@ func (d *decoder) handleValue(value *unstable.Node, v reflect.Value) error {
if d.unmarshalerInterface { if d.unmarshalerInterface {
if v.CanAddr() && v.Addr().CanInterface() { if v.CanAddr() && v.Addr().CanInterface() {
if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok { if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok {
return outi.UnmarshalTOML(value) // Pass raw bytes from the original document
return outi.UnmarshalTOML(d.p.Raw(value.Raw))
} }
} }
} }
@@ -1201,7 +1270,8 @@ func (d *decoder) handleKeyValuePart(key unstable.Iterator, value *unstable.Node
if d.unmarshalerInterface { if d.unmarshalerInterface {
if v.CanAddr() && v.Addr().CanInterface() { if v.CanAddr() && v.Addr().CanInterface() {
if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok { if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok {
return reflect.Value{}, outi.UnmarshalTOML(value) // Pass raw bytes from the original document
return reflect.Value{}, outi.UnmarshalTOML(d.p.Raw(value.Raw))
} }
} }
} }
+360 -449
View File
@@ -11,9 +11,9 @@ import (
"testing" "testing"
"time" "time"
"github.com/pelletier/go-toml/v2" "git.ostiwe.com/ostiwe/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
"github.com/pelletier/go-toml/v2/unstable" "git.ostiwe.com/ostiwe/go-toml/v2/unstable"
) )
type unmarshalTextKey struct { type unmarshalTextKey struct {
@@ -96,6 +96,132 @@ func ExampleUnmarshal() {
// tags: [go toml] // tags: [go toml]
} }
// pluginConfig demonstrates how to implement dynamic unmarshaling
// based on a "type" field. This pattern is useful for plugin systems
// or polymorphic configuration.
type pluginConfig struct {
Type string
Config any
}
func (p *pluginConfig) UnmarshalTOML(data []byte) error {
// First, decode just the type field
var typeOnly struct {
Type string `toml:"type"`
}
if err := toml.Unmarshal(data, &typeOnly); err != nil {
return err
}
p.Type = typeOnly.Type
// Now decode the config based on the type
switch typeOnly.Type {
case "database":
var cfg struct {
Type string `toml:"type"`
Host string `toml:"host"`
Port int `toml:"port"`
}
if err := toml.Unmarshal(data, &cfg); err != nil {
return err
}
p.Config = map[string]any{"host": cfg.Host, "port": cfg.Port}
case "cache":
var cfg struct {
Type string `toml:"type"`
TTL int `toml:"ttl"`
}
if err := toml.Unmarshal(data, &cfg); err != nil {
return err
}
p.Config = map[string]any{"ttl": cfg.TTL}
}
return nil
}
// This example demonstrates dynamic unmarshaling based on a discriminator
// field. The pluginConfig type uses UnmarshalTOML to first read the "type"
// field, then decode the rest of the configuration based on that type.
// This pattern is useful for plugin systems or configuration that varies
// by type.
func ExampleDecoder_EnableUnmarshalerInterface_dynamicConfig() {
doc := `
[[plugins]]
type = "database"
host = "localhost"
port = 5432
[[plugins]]
type = "cache"
ttl = 300
`
type Config struct {
Plugins []pluginConfig `toml:"plugins"`
}
var cfg Config
err := toml.NewDecoder(strings.NewReader(doc)).
EnableUnmarshalerInterface().
Decode(&cfg)
if err != nil {
panic(err)
}
for _, p := range cfg.Plugins {
fmt.Printf("type=%s config=%v\n", p.Type, p.Config)
}
// Output:
// type=database config=map[host:localhost port:5432]
// type=cache config=map[ttl:300]
}
// This example demonstrates using RawMessage to capture raw TOML bytes
// for later processing. RawMessage is similar to json.RawMessage - it
// delays decoding so you can inspect the raw content or decode it
// differently based on context.
func ExampleDecoder_EnableUnmarshalerInterface_rawMessage() {
doc := `
[plugin]
name = "example"
version = "1.0"
enabled = true
`
type Config struct {
Plugin unstable.RawMessage `toml:"plugin"`
}
var cfg Config
err := toml.NewDecoder(strings.NewReader(doc)).
EnableUnmarshalerInterface().
Decode(&cfg)
if err != nil {
panic(err)
}
// cfg.Plugin contains the raw TOML bytes
fmt.Printf("Raw TOML captured:\n%s", cfg.Plugin)
// You can later decode it into a specific type
var plugin struct {
Name string `toml:"name"`
Version string `toml:"version"`
Enabled bool `toml:"enabled"`
}
if err := toml.Unmarshal(cfg.Plugin, &plugin); err != nil {
panic(err)
}
fmt.Printf("Decoded: name=%s version=%s enabled=%v\n",
plugin.Name, plugin.Version, plugin.Enabled)
// Output:
// Raw TOML captured:
// name = "example"
// version = "1.0"
// enabled = true
// Decoded: name=example version=1.0 enabled=true
}
type badReader struct{} type badReader struct{}
func (r *badReader) Read([]byte) (int, error) { func (r *badReader) Read([]byte) (int, error) {
@@ -600,96 +726,6 @@ foo = "bar"`,
} }
}, },
}, },
{
desc: "local-time without seconds",
input: `a = 14:15`,
gen: func() test {
var v map[string]interface{}
return test{
target: &v,
expected: &map[string]interface{}{
"a": toml.LocalTime{Hour: 14, Minute: 15},
},
}
},
},
{
desc: "local-datetime without seconds using T",
input: `a = 2010-02-03T14:15`,
gen: func() test {
var v map[string]interface{}
return test{
target: &v,
expected: &map[string]interface{}{
"a": toml.LocalDateTime{
LocalDate: toml.LocalDate{2010, 2, 3},
LocalTime: toml.LocalTime{Hour: 14, Minute: 15},
},
},
}
},
},
{
desc: "local-datetime without seconds using space",
input: `a = 2010-02-03 14:15`,
gen: func() test {
var v map[string]interface{}
return test{
target: &v,
expected: &map[string]interface{}{
"a": toml.LocalDateTime{
LocalDate: toml.LocalDate{2010, 2, 3},
LocalTime: toml.LocalTime{Hour: 14, Minute: 15},
},
},
}
},
},
{
desc: "datetime without seconds with Z",
input: `a = 2010-02-03T14:15Z`,
gen: func() test {
var v map[string]time.Time
return test{
target: &v,
expected: &map[string]time.Time{
"a": time.Date(2010, 2, 3, 14, 15, 0, 0, time.UTC),
},
}
},
},
{
desc: "datetime without seconds with offset",
input: `a = 2010-02-03T14:15+05:00`,
gen: func() test {
var v map[string]time.Time
return test{
target: &v,
expected: &map[string]time.Time{
"a": time.Date(2010, 2, 3, 14, 15, 0, 0, time.FixedZone("", 5*3600)),
},
}
},
},
{
desc: "local-time with seconds and fractional regression",
input: `a = 14:15:30.123`,
gen: func() test {
var v map[string]interface{}
return test{
target: &v,
expected: &map[string]interface{}{
"a": toml.LocalTime{Hour: 14, Minute: 15, Second: 30, Nanosecond: 123000000, Precision: 3},
},
}
},
},
{ {
desc: "local-time missing digit", desc: "local-time missing digit",
input: `a = 12:08:0`, input: `a = 12:08:0`,
@@ -849,104 +885,6 @@ huey = 'dewey'
} }
}, },
}, },
{
desc: "basic string escape character",
input: `A = "\e"`,
gen: func() test {
type doc struct {
A string
}
return test{
target: &doc{},
expected: &doc{A: "\x1B"},
}
},
},
{
desc: "multiline basic string escape character",
input: `A = """\e"""`,
gen: func() test {
type doc struct {
A string
}
return test{
target: &doc{},
expected: &doc{A: "\x1B"},
}
},
},
{
desc: "escape character combined with bracket",
input: `A = "\e["`,
gen: func() test {
type doc struct {
A string
}
return test{
target: &doc{},
expected: &doc{A: "\x1B["},
}
},
},
{
desc: "basic string hex escape lowercase letter",
input: `A = "\x61"`,
gen: func() test {
type doc struct {
A string
}
return test{
target: &doc{},
expected: &doc{A: "a"},
}
},
},
{
desc: "basic string hex escape null byte",
input: `A = "\x00"`,
gen: func() test {
type doc struct {
A string
}
return test{
target: &doc{},
expected: &doc{A: "\x00"},
}
},
},
{
desc: "basic string hex escape max value",
input: `A = "\xFF"`,
gen: func() test {
type doc struct {
A string
}
return test{
target: &doc{},
expected: &doc{A: "\u00FF"},
}
},
},
{
desc: "multiline basic string hex escape",
input: `A = """\x61"""`,
gen: func() test {
type doc struct {
A string
}
return test{
target: &doc{},
expected: &doc{A: "a"},
}
},
},
{ {
desc: "spaces around dotted keys", desc: "spaces around dotted keys",
input: "a . b = 1", input: "a . b = 1",
@@ -1094,87 +1032,6 @@ B = "data"`,
} }
}, },
}, },
{
desc: "multiline inline table",
input: "Name = {\n First = \"hello\",\n Last = \"world\"\n}",
gen: func() test {
type name struct {
First string
Last string
}
type doc struct {
Name name
}
return test{
target: &doc{},
expected: &doc{Name: name{
First: "hello",
Last: "world",
}},
}
},
},
{
desc: "inline table with trailing comma",
input: `Name = {First = "hello", Last = "world",}`,
gen: func() test {
type name struct {
First string
Last string
}
type doc struct {
Name name
}
return test{
target: &doc{},
expected: &doc{Name: name{
First: "hello",
Last: "world",
}},
}
},
},
{
desc: "multiline inline table with trailing comma and comments",
input: "Name = {\n # first name\n First = \"hello\",\n # last name\n Last = \"world\",\n}",
gen: func() test {
type name struct {
First string
Last string
}
type doc struct {
Name name
}
return test{
target: &doc{},
expected: &doc{Name: name{
First: "hello",
Last: "world",
}},
}
},
},
{
desc: "nested multiline inline tables",
input: "A = {\n B = {\n C = 1,\n },\n}",
gen: func() test {
var v map[string]interface{}
return test{
target: &v,
expected: &map[string]interface{}{
"A": map[string]interface{}{
"B": map[string]interface{}{
"C": int64(1),
},
},
},
}
},
},
{ {
desc: "inline table inside array", desc: "inline table inside array",
input: `Names = [{First = "hello", Last = "world"}, {First = "ab", Last = "cd"}]`, input: `Names = [{First = "hello", Last = "world"}, {First = "ab", Last = "cd"}]`,
@@ -3414,7 +3271,7 @@ world'`,
{ {
desc: "bad char between minutes and seconds", desc: "bad char between minutes and seconds",
data: `a = 2021-03-30 21:312:0`, data: `a = 2021-03-30 21:312:0`,
msg: `extra characters at the end of a local date time`, msg: `expecting colon between minutes and seconds`,
}, },
{ {
desc: "invalid hour value", desc: "invalid hour value",
@@ -3529,18 +3386,6 @@ world'`,
desc: `invalid escape char basic multiline string`, desc: `invalid escape char basic multiline string`,
data: `A = """\z"""`, data: `A = """\z"""`,
}, },
{
desc: `invalid hex escape non-hex character in basic string`,
data: `A = "\xGG"`,
},
{
desc: `incomplete hex escape in basic string`,
data: `A = "\x6"`,
},
{
desc: `invalid hex escape non-hex character in multiline basic string`,
data: `A = """\xGG"""`,
},
{ {
desc: `invalid inf`, desc: `invalid inf`,
data: `A = ick`, data: `A = ick`,
@@ -3727,30 +3572,6 @@ world'`,
desc: `backspace in comment`, desc: `backspace in comment`,
data: "# this is a test\ba=1", data: "# this is a test\ba=1",
}, },
{
desc: `inline table comma at start`,
data: `a = { , b = 1 }`,
},
{
desc: `inline table missing separator`,
data: `a = { b = 1 c = 2 }`,
},
{
desc: `inline table double comma across newline`,
data: "a = { b = 1,\n, c = 2 }",
},
{
desc: `incomplete inline table`,
data: "a = { b = 1,\n",
},
{
desc: `incomplete hex escape in multiline basic string`,
data: `A = """\x6"""`,
},
{
desc: `invalid escape char in basic string`,
data: `A = "\z"`,
},
} }
for _, e := range examples { for _, e := range examples {
@@ -4205,8 +4026,8 @@ type CustomUnmarshalerKey struct {
A int64 A int64
} }
func (k *CustomUnmarshalerKey) UnmarshalTOML(value *unstable.Node) error { func (k *CustomUnmarshalerKey) UnmarshalTOML(data []byte) error {
item, err := strconv.ParseInt(string(value.Data), 10, 64) item, err := strconv.ParseInt(string(data), 10, 64)
if err != nil { if err != nil {
return fmt.Errorf("error converting to int64, %w", err) return fmt.Errorf("error converting to int64, %w", err)
} }
@@ -4294,7 +4115,7 @@ foo = "bar"`,
type doc994 struct{} type doc994 struct{}
func (d *doc994) UnmarshalTOML(*unstable.Node) error { func (d *doc994) UnmarshalTOML([]byte) error {
return errors.New("expected-error") return errors.New("expected-error")
} }
@@ -4317,8 +4138,8 @@ type doc994ok struct {
S string S string
} }
func (d *doc994ok) UnmarshalTOML(value *unstable.Node) error { func (d *doc994ok) UnmarshalTOML(data []byte) error {
d.S = string(value.Data) + " from unmarshaler" d.S = string(data) + " from unmarshaler"
return nil return nil
} }
@@ -4331,7 +4152,8 @@ func TestIssue994_OK(t *testing.T) {
Decode(&d) Decode(&d)
assert.NoError(t, err) assert.NoError(t, err)
assert.Equal(t, "bar from unmarshaler", d.S) // With bytes-based interface, raw TOML bytes are passed including quotes
assert.Equal(t, "\"bar\" from unmarshaler", d.S)
} }
func TestIssue995(t *testing.T) { func TestIssue995(t *testing.T) {
@@ -4691,175 +4513,264 @@ func TestIssue1028(t *testing.T) {
}) })
} }
// customFieldUnmarshaler implements unstable.Unmarshaler and captures all // Tests for issue #873 - Bring back toml.Unmarshaler for tables and arrays
// key-value pairs directed to it, including unknown fields.
type customFieldUnmarshaler struct { type customTable873 struct {
Keys []string
Values map[string]string Values map[string]string
} }
func (c *customFieldUnmarshaler) UnmarshalTOML(value *unstable.Node) error { func (c *customTable873) UnmarshalTOML(data []byte) error {
c.Values = map[string]string{ c.Keys = []string{}
"kind": value.Kind.String(), c.Values = make(map[string]string)
"data": string(value.Data),
// Parse the raw TOML bytes into a map to extract keys in order
// For this test, we use a simple line-by-line parser to preserve order
lines := bytes.Split(data, []byte{'\n'})
for _, line := range lines {
line = bytes.TrimSpace(line)
if len(line) == 0 {
continue
}
// Skip table headers
if line[0] == '[' {
continue
}
// Parse key = value
eqIdx := bytes.Index(line, []byte{'='})
if eqIdx < 0 {
continue
}
key := string(bytes.TrimSpace(line[:eqIdx]))
// Remove quotes from quoted keys
if len(key) >= 2 && key[0] == '"' && key[len(key)-1] == '"' {
key = key[1 : len(key)-1]
}
valueBytes := bytes.TrimSpace(line[eqIdx+1:])
// Remove quotes from string values
if len(valueBytes) >= 2 && valueBytes[0] == '"' && valueBytes[len(valueBytes)-1] == '"' {
valueBytes = valueBytes[1 : len(valueBytes)-1]
}
c.Keys = append(c.Keys, key)
c.Values[key] = string(valueBytes)
} }
return nil return nil
} }
func TestUnmarshalerInterface_StructFieldFallback(t *testing.T) { // Test for split tables - when the same parent table is defined in multiple places
// When EnableUnmarshalerInterface is active and a struct field is not found, // This is a key requirement for issue #873: if type A implements Unmarshaler,
// the decoder should fall back to the Unmarshaler interface on the struct. // and [a.b] and [a.d] are defined with another table [x] in between,
// A should receive content for both b and d, but not x.
func TestIssue873_SplitTables(t *testing.T) {
// For this test, we expect each sub-table to be handled separately
// The parent doesn't receive the sub-tables directly - each sub-table
// (b and d) gets its own call to handleKeyValues
type Config struct { type Config struct {
Name string `toml:"name"` A struct {
B customTable873 `toml:"b"`
D customTable873 `toml:"d"`
} `toml:"a"`
X customTable873 `toml:"x"`
} }
t.Run("unknown field with unmarshaler", func(t *testing.T) { doc := `
doc := `name = "hello" [a.b]
unknown = "world"` C = "1"
var cfg Config
decoder := toml.NewDecoder(bytes.NewReader([]byte(doc)))
decoder.EnableUnmarshalerInterface()
err := decoder.Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, "hello", cfg.Name)
})
}
func TestUnmarshalerInterface_Value(t *testing.T) { [x]
// Test that EnableUnmarshalerInterface delegates value decoding Y = "100"
// to the UnmarshalTOML method.
type Config struct { [a.d]
Field customFieldUnmarshaler `toml:"field"` E = "2"
} `
doc := `field = "test-value"`
var cfg Config var cfg Config
decoder := toml.NewDecoder(bytes.NewReader([]byte(doc))) err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
decoder.EnableUnmarshalerInterface() EnableUnmarshalerInterface().
err := decoder.Decode(&cfg) Decode(&cfg)
assert.NoError(t, err) assert.NoError(t, err)
assert.Equal(t, "test-value", cfg.Field.Values["data"]) // Each sub-table should have received its own key-values
assert.Equal(t, []string{"C"}, cfg.A.B.Keys)
assert.Equal(t, "1", cfg.A.B.Values["C"])
assert.Equal(t, []string{"E"}, cfg.A.D.Keys)
assert.Equal(t, "2", cfg.A.D.Values["E"])
assert.Equal(t, []string{"Y"}, cfg.X.Keys)
assert.Equal(t, "100", cfg.X.Values["Y"])
} }
func TestTypeMismatchString_StructFieldContext(t *testing.T) { // Test using RawMessage to capture raw TOML bytes
// Exercise the typeMismatchString code path that includes struct field info func TestIssue873_RawMessage(t *testing.T) {
// in the error message.
type Inner struct {
Value int `toml:"value"`
}
type Config struct { type Config struct {
Inner Inner `toml:"inner"` Plugin unstable.RawMessage `toml:"plugin"`
} }
doc := `inner = "not-a-table"` doc := `
[plugin]
name = "example"
version = "1.0"
`
var cfg Config var cfg Config
err := toml.Unmarshal([]byte(doc), &cfg) err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
// RawMessage should contain the raw key-value bytes
expected := "name = \"example\"\nversion = \"1.0\"\n"
assert.Equal(t, expected, string(cfg.Plugin))
}
// Test keys that need quoting (contain special characters)
func TestIssue873_QuotedKeys(t *testing.T) {
type Config struct {
Section customTable873 `toml:"section"`
}
doc := `
[section]
"key with spaces" = "value1"
"key.with.dots" = "value2"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, 2, len(cfg.Section.Keys))
assert.Equal(t, "value1", cfg.Section.Values["key with spaces"])
assert.Equal(t, "value2", cfg.Section.Values["key.with.dots"])
}
// errorUnmarshaler873 is used to test error propagation from UnmarshalTOML
type errorUnmarshaler873 struct{}
func (e *errorUnmarshaler873) UnmarshalTOML([]byte) error {
return errors.New("intentional error")
}
// Test error propagation from UnmarshalTOML
func TestIssue873_UnmarshalerError(t *testing.T) {
doc := `
[section]
key = "value"
`
type Config struct {
Section errorUnmarshaler873 `toml:"section"`
}
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.Error(t, err) assert.Error(t, err)
assert.True(t, strings.Contains(err.Error(), "intentional error"))
} }
func TestUnmarshalInlineTable_IncompatibleType(t *testing.T) { // Test dotted keys in a table (e.g., a.b = value)
// Exercise the default branch of unmarshalInlineTable when the target func TestIssue873_DottedKeys(t *testing.T) {
// is not a map, struct, or interface. type Config struct {
type doc struct { Section customTable873 `toml:"section"`
A int `toml:"a"`
} }
var v doc
err := toml.Unmarshal([]byte(`a = {b = 1}`), &v)
assert.Error(t, err)
}
func TestTypeMismatchString_NoStructContext(t *testing.T) { doc := `
// Exercise the typeMismatchString code path without struct field context (line 186). [section]
// Decoding a string into a bare int triggers this path. sub.key = "value1"
var v map[string]int another.nested.key = "value2"
err := toml.Unmarshal([]byte(`a = "hello"`), &v) `
assert.Error(t, err)
} var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
func TestMultilineInlineTable_EmptyWithNewlines(t *testing.T) {
doc := "a = {\n\n}"
var v map[string]interface{}
err := toml.Unmarshal([]byte(doc), &v)
assert.NoError(t, err) assert.NoError(t, err)
inner := v["a"] assert.Equal(t, 2, len(cfg.Section.Keys))
if inner == nil { // The dotted keys should be preserved in the raw output
t.Fatal("expected key 'a' to be present") assert.Equal(t, "value1", cfg.Section.Values["sub.key"])
} assert.Equal(t, "value2", cfg.Section.Values["another.nested.key"])
m, ok := inner.(map[string]interface{})
if !ok {
t.Fatalf("expected map, got %T", inner)
}
if len(m) != 0 {
t.Fatalf("expected empty map, got %v", m)
}
} }
func TestMultilineInlineTable_CommentsOnly(t *testing.T) { // Test pointer to pointer to Unmarshaler (covers pointer dereferencing loop)
doc := "a = {\n # just a comment\n}" func TestIssue873_DoublePointerUnmarshaler(t *testing.T) {
var v map[string]interface{} type Config struct {
err := toml.Unmarshal([]byte(doc), &v) Section **customTable873 `toml:"section"`
}
doc := `
[section]
key = "value"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err) assert.NoError(t, err)
inner := v["a"] assert.True(t, cfg.Section != nil)
if inner == nil { assert.True(t, *cfg.Section != nil)
t.Fatal("expected key 'a' to be present") assert.Equal(t, []string{"key"}, (*cfg.Section).Keys)
} assert.Equal(t, "value", (*cfg.Section).Values["key"])
m, ok := inner.(map[string]interface{})
if !ok {
t.Fatalf("expected map, got %T", inner)
}
if len(m) != 0 {
t.Fatalf("expected empty map, got %v", m)
}
} }
func TestMultilineInlineTable_CommentAfterComma(t *testing.T) { // formattingCapture captures the raw TOML bytes to verify formatting preservation
// Exercises comment handling after comma in inline table (parser lines 518-524). type formattingCapture struct {
doc := "a = { b = 1, # comment\nc = 2 }" RawBytes string
var v map[string]interface{}
err := toml.Unmarshal([]byte(doc), &v)
assert.NoError(t, err)
m, ok := v["a"].(map[string]interface{})
if !ok {
t.Fatal("expected a map")
}
if m["b"] != int64(1) {
t.Fatalf("expected b=1, got %v", m["b"])
}
if m["c"] != int64(2) {
t.Fatalf("expected c=2, got %v", m["c"])
}
} }
func TestMultilineInlineTable_CommentAfterValue(t *testing.T) { func (f *formattingCapture) UnmarshalTOML(data []byte) error {
// Exercises comment handling after keyval in inline table (parser lines 542-548). f.RawBytes = string(data)
doc := "a = { b = 1 # comment\n, c = 2 }" return nil
var v map[string]interface{}
err := toml.Unmarshal([]byte(doc), &v)
assert.NoError(t, err)
m, ok := v["a"].(map[string]interface{})
if !ok {
t.Fatal("expected a map")
}
if m["b"] != int64(1) {
t.Fatalf("expected b=1, got %v", m["b"])
}
if m["c"] != int64(2) {
t.Fatalf("expected c=2, got %v", m["c"])
}
} }
func TestMultilineInlineTable_LeadingComma(t *testing.T) { func TestIssue873_FormattingPreservation(t *testing.T) {
doc := "a = { b = 1\n, c = 2 }" type Config struct {
var v map[string]interface{} Section *formattingCapture `toml:"section"`
err := toml.Unmarshal([]byte(doc), &v) }
// Test that various formatting styles are preserved:
// - Extra spaces around '='
// - Literal strings (single quotes)
// - Hex numbers
// - Inline tables
doc := `[section]
key1 = "value with spaces"
key2 = 'literal string'
hex_val = 0xDEADBEEF
inline = { a = 1, b = 2 }
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err) assert.NoError(t, err)
m, ok := v["a"].(map[string]interface{}) assert.True(t, cfg.Section != nil)
if !ok {
t.Fatal("expected a map") // The raw bytes should preserve original formatting
} raw := cfg.Section.RawBytes
if m["b"] != int64(1) {
t.Fatalf("expected b=1, got %v", m["b"]) // Check that extra spaces around '=' are preserved
} assert.True(t, strings.Contains(raw, "key1 = \"value with spaces\""),
if m["c"] != int64(2) { "Expected spacing to be preserved, got: %s", raw)
t.Fatalf("expected c=2, got %v", m["c"])
} // Check that literal string style is preserved
assert.True(t, strings.Contains(raw, "key2 = 'literal string'"),
"Expected literal string to be preserved, got: %s", raw)
// Check that hex format is preserved
assert.True(t, strings.Contains(raw, "hex_val = 0xDEADBEEF"),
"Expected hex format to be preserved, got: %s", raw)
// Check that inline table is preserved
assert.True(t, strings.Contains(raw, "inline = { a = 1, b = 2 }"),
"Expected inline table to be preserved, got: %s", raw)
} }
+7 -3
View File
@@ -28,12 +28,16 @@ func (c *Iterator) Next() bool {
if c.nodes == nil { if c.nodes == nil {
return false return false
} }
nodes := *c.nodes
if !c.started { if !c.started {
c.started = true c.started = true
} else if c.idx >= 0 { } else {
c.idx = (*c.nodes)[c.idx].next idx := c.idx
if idx >= 0 && int(idx) < len(nodes) {
c.idx = nodes[idx].next
}
} }
return c.idx >= 0 && int(c.idx) < len(*c.nodes) return c.idx >= 0 && int(c.idx) < len(nodes)
} }
// IsLast returns true if the current node of the iterator is the last // IsLast returns true if the current node of the iterator is the last
+24 -64
View File
@@ -5,7 +5,7 @@ import (
"fmt" "fmt"
"unicode" "unicode"
"github.com/pelletier/go-toml/v2/internal/characters" "git.ostiwe.com/ostiwe/go-toml/v2/internal/characters"
) )
// ParserError describes an error relative to the content of the document. // ParserError describes an error relative to the content of the document.
@@ -328,6 +328,9 @@ func (p *Parser) parseStdTable(b []byte) (reference, []byte, error) {
func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) { func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) {
// keyval = key keyval-sep val // keyval = key keyval-sep val
// Track the start position for Raw range
startB := b
ref := p.builder.Push(Node{ ref := p.builder.Push(Node{
Kind: KeyValue, Kind: KeyValue,
}) })
@@ -342,7 +345,7 @@ func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) {
b = p.parseWhitespace(b) b = p.parseWhitespace(b)
if len(b) == 0 { if len(b) == 0 {
return invalidReference, nil, NewParserError(b, "expected = after a key, but the document ends there") return invalidReference, nil, NewParserError(startB[:len(startB)-len(b)], "expected = after a key, but the document ends there")
} }
b, err = expect('=', b) b, err = expect('=', b)
@@ -360,6 +363,11 @@ func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) {
p.builder.Chain(valRef, key) p.builder.Chain(valRef, key)
p.builder.AttachChild(ref, valRef) p.builder.AttachChild(ref, valRef)
// Set Raw to span the entire key-value expression.
// Access the node directly in the slice to avoid the write barrier
// that NodeAt's nodes-pointer setup would trigger.
p.builder.tree.nodes[ref].Raw = p.rangeOfToken(startB[:len(startB)-len(b)], b)
return ref, b, err return ref, b, err
} }
@@ -460,14 +468,12 @@ func (p *Parser) parseLiteralString(b []byte) ([]byte, []byte, []byte, error) {
return v, v[1 : len(v)-1], rest, nil return v, v[1 : len(v)-1], rest, nil
} }
//nolint:funlen,cyclop,dupl
func (p *Parser) parseInlineTable(b []byte) (reference, []byte, error) { func (p *Parser) parseInlineTable(b []byte) (reference, []byte, error) {
// inline-table = inline-table-open [ inline-table-keyvals ] inline-table-close // inline-table = inline-table-open [ inline-table-keyvals ] inline-table-close
// inline-table-open = %x7B ws ; { // inline-table-open = %x7B ws ; {
// inline-table-close = ws %x7D ; } // inline-table-close = ws %x7D ; }
// inline-table-sep = ws %x2C ws ; , Comma // inline-table-sep = ws %x2C ws ; , Comma
// inline-table-keyvals = keyval [ inline-table-sep inline-table-keyvals ] // inline-table-keyvals = keyval [ inline-table-sep inline-table-keyvals ]
tableStart := b
parent := p.builder.Push(Node{ parent := p.builder.Push(Node{
Kind: InlineTable, Kind: InlineTable,
Raw: p.rangeOfToken(b[:1], b[1:]), Raw: p.rangeOfToken(b[:1], b[1:]),
@@ -475,77 +481,45 @@ func (p *Parser) parseInlineTable(b []byte) (reference, []byte, error) {
first := true first := true
lastChild := invalidReference var child reference
addChild := func(ref reference) {
if lastChild == invalidReference {
p.builder.AttachChild(parent, ref)
} else {
p.builder.Chain(lastChild, ref)
}
lastChild = ref
}
b = b[1:] b = b[1:]
var err error var err error
for len(b) > 0 { for len(b) > 0 {
var cref reference previousB := b
cref, b, err = p.parseOptionalWhitespaceCommentNewline(b) b = p.parseWhitespace(b)
if err != nil {
return parent, nil, err
}
if cref != invalidReference {
addChild(cref)
}
if len(b) == 0 { if len(b) == 0 {
return parent, nil, NewParserError(tableStart[:1], "inline table is incomplete") return parent, nil, NewParserError(previousB[:1], "inline table is incomplete")
} }
if b[0] == '}' { if b[0] == '}' {
break break
} }
if b[0] == ',' { if !first {
if first { b, err = expect(',', b)
return parent, nil, NewParserError(b[0:1], "inline table cannot start with comma")
}
b = b[1:]
cref, b, err = p.parseOptionalWhitespaceCommentNewline(b)
if err != nil { if err != nil {
return parent, nil, err return parent, nil, err
} }
if cref != invalidReference { b = p.parseWhitespace(b)
addChild(cref)
}
} else if !first {
return parent, nil, NewParserError(b[0:1], "inline table entries must be separated by commas")
}
// trailing comma: if '}' follows, stop
if len(b) > 0 && b[0] == '}' {
break
} }
var kv reference var kv reference
kv, b, err = p.parseKeyval(b) kv, b, err = p.parseKeyval(b)
if err != nil { if err != nil {
return parent, nil, err return parent, nil, err
} }
addChild(kv) if first {
p.builder.AttachChild(parent, kv)
cref, b, err = p.parseOptionalWhitespaceCommentNewline(b) } else {
if err != nil { p.builder.Chain(child, kv)
return parent, nil, err
}
if cref != invalidReference {
addChild(cref)
} }
child = kv
first = false first = false
} }
@@ -555,7 +529,7 @@ func (p *Parser) parseInlineTable(b []byte) (reference, []byte, error) {
return parent, rest, err return parent, rest, err
} }
//nolint:funlen,cyclop,dupl //nolint:funlen,cyclop
func (p *Parser) parseValArray(b []byte) (reference, []byte, error) { func (p *Parser) parseValArray(b []byte) (reference, []byte, error) {
// array = array-open [ array-values ] ws-comment-newline array-close // array = array-open [ array-values ] ws-comment-newline array-close
// array-open = %x5B ; [ // array-open = %x5B ; [
@@ -819,13 +793,6 @@ func (p *Parser) parseMultilineBasicString(b []byte) ([]byte, []byte, []byte, er
builder.WriteByte('\t') builder.WriteByte('\t')
case 'e': case 'e':
builder.WriteByte(0x1B) builder.WriteByte(0x1B)
case 'x':
x, err := hexToRune(atmost(token[i+1:], 2), 2)
if err != nil {
return nil, nil, nil, err
}
builder.WriteRune(x)
i += 2
case 'u': case 'u':
x, err := hexToRune(atmost(token[i+1:], 4), 4) x, err := hexToRune(atmost(token[i+1:], 4), 4)
if err != nil { if err != nil {
@@ -985,13 +952,6 @@ func (p *Parser) parseBasicString(b []byte) ([]byte, []byte, []byte, error) {
builder.WriteByte('\t') builder.WriteByte('\t')
case 'e': case 'e':
builder.WriteByte(0x1B) builder.WriteByte(0x1B)
case 'x':
x, err := hexToRune(token[i+1:len(token)-1], 2)
if err != nil {
return nil, nil, nil, err
}
builder.WriteRune(x)
i += 2
case 'u': case 'u':
x, err := hexToRune(token[i+1:len(token)-1], 4) x, err := hexToRune(token[i+1:len(token)-1], 4)
if err != nil { if err != nil {
+7 -193
View File
@@ -6,7 +6,7 @@ import (
"strings" "strings"
"testing" "testing"
"github.com/pelletier/go-toml/v2/internal/assert" "git.ostiwe.com/ostiwe/go-toml/v2/internal/assert"
) )
func TestParser_AST_Numbers(t *testing.T) { func TestParser_AST_Numbers(t *testing.T) {
@@ -331,154 +331,6 @@ func TestParser_AST(t *testing.T) {
}, },
}, },
}, },
{
desc: "multiline inline table",
input: "name = {\n first = \"Tom\",\n last = \"Preston-Werner\"\n}",
ast: astNode{
Kind: KeyValue,
Children: []astNode{
{
Kind: InlineTable,
Children: []astNode{
{
Kind: KeyValue,
Children: []astNode{
{Kind: String, Data: []byte(`Tom`)},
{Kind: Key, Data: []byte(`first`)},
},
},
{
Kind: KeyValue,
Children: []astNode{
{Kind: String, Data: []byte(`Preston-Werner`)},
{Kind: Key, Data: []byte(`last`)},
},
},
},
},
{
Kind: Key,
Data: []byte(`name`),
},
},
},
},
{
desc: "inline table with trailing comma",
input: `name = { first = "Tom", last = "Preston-Werner", }`,
ast: astNode{
Kind: KeyValue,
Children: []astNode{
{
Kind: InlineTable,
Children: []astNode{
{
Kind: KeyValue,
Children: []astNode{
{Kind: String, Data: []byte(`Tom`)},
{Kind: Key, Data: []byte(`first`)},
},
},
{
Kind: KeyValue,
Children: []astNode{
{Kind: String, Data: []byte(`Preston-Werner`)},
{Kind: Key, Data: []byte(`last`)},
},
},
},
},
{
Kind: Key,
Data: []byte(`name`),
},
},
},
},
{
desc: "empty inline table with newline",
input: "name = {\n}",
ast: astNode{
Kind: KeyValue,
Children: []astNode{
{
Kind: InlineTable,
Children: nil,
},
{
Kind: Key,
Data: []byte(`name`),
},
},
},
},
{
desc: "inline table with leading comma",
input: "name = { first = \"Tom\"\n, last = \"Werner\" }",
ast: astNode{
Kind: KeyValue,
Children: []astNode{
{
Kind: InlineTable,
Children: []astNode{
{
Kind: KeyValue,
Children: []astNode{
{Kind: String, Data: []byte(`Tom`)},
{Kind: Key, Data: []byte(`first`)},
},
},
{
Kind: KeyValue,
Children: []astNode{
{Kind: String, Data: []byte(`Werner`)},
{Kind: Key, Data: []byte(`last`)},
},
},
},
},
{
Kind: Key,
Data: []byte(`name`),
},
},
},
},
{
desc: "inline table with leading trailing comma",
input: "name = { first = \"Tom\"\n, }",
ast: astNode{
Kind: KeyValue,
Children: []astNode{
{
Kind: InlineTable,
Children: []astNode{
{
Kind: KeyValue,
Children: []astNode{
{Kind: String, Data: []byte(`Tom`)},
{Kind: Key, Data: []byte(`first`)},
},
},
},
},
{
Kind: Key,
Data: []byte(`name`),
},
},
},
},
{
desc: "inline table comma at start is error",
input: "name = { , first = \"Tom\" }",
err: true,
},
{
desc: "inline table double comma across newline is error",
input: "name = { first = \"Tom\",\n, last = \"Werner\" }",
err: true,
},
} }
for _, e := range examples { for _, e := range examples {
@@ -498,44 +350,6 @@ func TestParser_AST(t *testing.T) {
} }
} }
func TestParseInlineTable_CommentsWithKeepComments(t *testing.T) {
// Exercise comment reference handling inside parseInlineTable when
// KeepComments is true. This covers the addChild(cref) branches
// at the start of the loop, after comma, and after keyval.
examples := []struct {
desc string
input string
}{
{
desc: "comment at start of inline table",
input: "a = {\n# comment\nb = 1\n}",
},
{
desc: "comment after comma",
input: "a = {b = 1,\n# comment\nc = 2\n}",
},
{
desc: "comment after keyval",
input: "a = {b = 1 # comment\n, c = 2}",
},
{
desc: "comment only in inline table",
input: "a = {\n# just a comment\n}",
},
}
for _, e := range examples {
e := e
t.Run(e.desc, func(t *testing.T) {
p := Parser{KeepComments: true}
p.Reset([]byte(e.input))
p.NextExpression()
err := p.Error()
assert.NoError(t, err)
})
}
}
func BenchmarkParseBasicStringWithUnicode(b *testing.B) { func BenchmarkParseBasicStringWithUnicode(b *testing.B) {
p := &Parser{} p := &Parser{}
b.Run("4", func(b *testing.B) { b.Run("4", func(b *testing.B) {
@@ -725,7 +539,7 @@ key5 = [ # Next to start of inline array.
// --- // ---
// 6:1->6:22 (105->126) | Comment [# Above simple value.] // 6:1->6:22 (105->126) | Comment [# Above simple value.]
// --- // ---
// 1:1->1:1 (0->0) | KeyValue [] // 7:1->7:14 (127->140) | KeyValue []
// 7:7->7:14 (133->140) | String [value] // 7:7->7:14 (133->140) | String [value]
// 7:1->7:4 (127->130) | Key [key] // 7:1->7:4 (127->130) | Key [key]
// 7:15->7:38 (141->164) | Comment [# Next to simple value.] // 7:15->7:38 (141->164) | Comment [# Next to simple value.]
@@ -738,12 +552,12 @@ key5 = [ # Next to start of inline array.
// --- // ---
// 14:1->14:22 (252->273) | Comment [# Above inline table.] // 14:1->14:22 (252->273) | Comment [# Above inline table.]
// --- // ---
// 1:1->1:1 (0->0) | KeyValue [] // 15:1->15:50 (274->323) | KeyValue []
// 15:8->15:9 (281->282) | InlineTable [] // 15:8->15:9 (281->282) | InlineTable []
// 1:1->1:1 (0->0) | KeyValue [] // 15:10->15:23 (283->296) | KeyValue []
// 15:18->15:23 (291->296) | String [Tom] // 15:18->15:23 (291->296) | String [Tom]
// 15:10->15:15 (283->288) | Key [first] // 15:10->15:15 (283->288) | Key [first]
// 1:1->1:1 (0->0) | KeyValue [] // 15:25->15:48 (298->321) | KeyValue []
// 15:32->15:48 (305->321) | String [Preston-Werner] // 15:32->15:48 (305->321) | String [Preston-Werner]
// 15:25->15:29 (298->302) | Key [last] // 15:25->15:29 (298->302) | Key [last]
// 15:1->15:5 (274->278) | Key [name] // 15:1->15:5 (274->278) | Key [name]
@@ -753,7 +567,7 @@ key5 = [ # Next to start of inline array.
// --- // ---
// 18:1->18:15 (371->385) | Comment [# Above array.] // 18:1->18:15 (371->385) | Comment [# Above array.]
// --- // ---
// 1:1->1:1 (0->0) | KeyValue [] // 19:1->19:20 (386->405) | KeyValue []
// 1:1->1:1 (0->0) | Array [] // 1:1->1:1 (0->0) | Array []
// 19:11->19:12 (396->397) | Integer [1] // 19:11->19:12 (396->397) | Integer [1]
// 19:14->19:15 (399->400) | Integer [2] // 19:14->19:15 (399->400) | Integer [2]
@@ -765,7 +579,7 @@ key5 = [ # Next to start of inline array.
// --- // ---
// 22:1->22:26 (448->473) | Comment [# Above multi-line array.] // 22:1->22:26 (448->473) | Comment [# Above multi-line array.]
// --- // ---
// 1:1->1:1 (0->0) | KeyValue [] // 23:1->31:2 (474->694) | KeyValue []
// 1:1->1:1 (0->0) | Array [] // 1:1->1:1 (0->0) | Array []
// 23:10->23:42 (483->515) | Comment [# Next to start of inline array.] // 23:10->23:42 (483->515) | Comment [# Next to start of inline array.]
// 24:3->24:38 (518->553) | Comment [# Second line before array content.] // 24:3->24:38 (518->553) | Comment [# Second line before array content.]
+1 -1
View File
@@ -1,6 +1,6 @@
package unstable package unstable
import "github.com/pelletier/go-toml/v2/internal/characters" import "git.ostiwe.com/ostiwe/go-toml/v2/internal/characters"
func scanFollows(b []byte, pattern string) bool { func scanFollows(b []byte, pattern string) bool {
n := len(pattern) n := len(pattern)
+28 -3
View File
@@ -1,7 +1,32 @@
package unstable package unstable
// The Unmarshaler interface may be implemented by types to customize their // Unmarshaler is implemented by types that can unmarshal a TOML
// behavior when being unmarshaled from a TOML document. // description of themselves. The input is a valid TOML document
// containing the relevant portion of the parsed document.
//
// For tables (including split tables defined in multiple places),
// the data contains the raw key-value bytes from the original document
// with adjusted table headers to be relative to the unmarshaling target.
type Unmarshaler interface { type Unmarshaler interface {
UnmarshalTOML(value *Node) error UnmarshalTOML(data []byte) error
}
// RawMessage is a raw encoded TOML value. It implements Unmarshaler
// and can be used to delay TOML decoding or capture raw content.
//
// Example usage:
//
// type Config struct {
// Plugin RawMessage `toml:"plugin"`
// }
//
// var cfg Config
// toml.NewDecoder(r).EnableUnmarshalerInterface().Decode(&cfg)
// // cfg.Plugin now contains the raw TOML bytes for [plugin]
type RawMessage []byte
// UnmarshalTOML implements Unmarshaler.
func (m *RawMessage) UnmarshalTOML(data []byte) error {
*m = append((*m)[0:0], data...)
return nil
} }