Compare commits

...

77 Commits

Author SHA1 Message Date
Cursor Agent 530b363c2f Remove all usages of unsafe
Removed all usages of `unsafe` and the `internal/danger` package from the codebase.

1.  **`unstable/ast.go`**: Refactored `Node` struct to use `*Node` pointers for `next` and `child` fields instead of integer offsets. This eliminates the need for `unsafe` pointer arithmetic in `Next()` and `Child()` methods.
2.  **`unstable/builder.go`**: Updated `builder` to manage pointers to nodes directly instead of integer offsets.
3.  **`unstable/parser.go`**:
    *   Replaced `danger.SubsliceOffset` with safe capacity-based calculation (`cap(p.data) - cap(b)`), which works because tokens are slices of the parser's input buffer.
4.  **`strict.go`** & **`errors.go`**: Replaced `danger.BytesRange` and `danger.SubsliceOffset` with safe slice capacity arithmetic.
5.  **`unmarshaler.go`**: Replaced `map[danger.TypeID]...` with `map[uintptr]...` for the field paths cache using `reflect.ValueOf(t).Pointer()`. This removes the need for `unsafe` access to `reflect.Type` internals.
6.  **`internal/tracker/seen_test.go`**: Replaced `unsafe.Sizeof` with `reflect.TypeOf(...).Size()`.
7.  **`internal/danger`**: Deleted the package entirely.

Benchmarks show a mix of performance changes:
- Small document unmarshaling (SimpleDocument/struct-4) got slower (+25%), likely due to pointer chasing vs contiguous array access.
- Large document unmarshaling (canada, citm, twitter) actually improved significantly (-24% to -45% latency), likely due to reduced allocation overhead or better cache locality in some paths.
- Memory usage for large datasets decreased significantly (-50% to -60% B/op).
- Overall geomean latency improved by ~6%.

No public interfaces were changed. All tests pass.
2026-01-04 13:24:24 +00:00
Cursor Agent f09f77ab06 Refactor unsafe pointer usage to use reflect.Type and pointers
Remove internal/danger package and replace unsafe pointer arithmetic with direct pointer manipulation. Update AST node references to use pointers instead of integer offsets. This improves code safety and maintainability.

Co-authored-by: thomas.pelletier <thomas.pelletier@bedrockrobotics.com>
2026-01-04 03:11:48 +00:00
Thomas Pelletier 9702fae9b8 Add AGENTS.md for AI agent contribution guidelines (#1017)
This file provides a concise summary of the contribution guidelines
from CONTRIBUTING.md, specifically tailored for AI agents working on
the codebase. It covers testing requirements, backward compatibility,
performance considerations, and code style expectations.

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-03 21:41:34 -05:00
Alexander Hecke 3cf1eb2312 improve Unmarshaling documentation (#1016) 2026-01-03 21:12:35 -05:00
Nathan Baulch 2af3554f90 Update toml-test to v1.6.0 (#1007) 2026-01-03 20:45:06 -05:00
dependabot[bot] 180c6ba2ba build(deps): bump actions/setup-go from 5 to 6 (#1002)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5 to 6.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-03 20:43:53 -05:00
dependabot[bot] dafc4173ef build(deps): bump github/codeql-action from 3 to 4 (#1006)
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3 to 4.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-03 20:43:43 -05:00
dependabot[bot] f1a83be671 build(deps): bump actions/upload-artifact from 4 to 6 (#1011)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 6.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-03 20:43:33 -05:00
dependabot[bot] 5aeb70b3f0 build(deps): bump actions/checkout from 5 to 6 (#1010)
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-03 20:43:20 -05:00
W. Michael Petullo 8384a5683c Use constant format strings with Printf-like functions (#1013)
Recent versions of Go object to the use of non-constant variables a
format strings. This commit fixes errors like this:

cli.go:26:47: non-constant format string in call to fmt.Fprintf

Signed-off-by: W. Michael Petullo <mike@flyn.org>
2026-01-03 20:42:58 -05:00
Étienne BERSAC 4369957cb4 Unwrap strict errors (#1012) 2025-12-21 16:20:24 +01:00
dependabot[bot] a0e8464967 build(deps): bump actions/checkout from 4 to 5 (#1001)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-26 09:53:08 +02:00
Nathan Baulch c57d0d559f Add omitzero tag support (#998) 2025-08-25 08:06:48 +02:00
Thomas Pelletier 644602b845 Script to test all versions of go (#1000) 2025-08-24 12:40:29 +02:00
Nathan Baulch 36df8eef6e General cleanup (#999) 2025-08-24 12:18:46 +02:00
Thomas Pelletier 18a2148713 Handle array table into an empty slice (#997)
Fix #995
2025-08-21 12:05:41 +02:00
Thomas Pelletier bc9958322f Add missing UnmarshalTOML call (#996)
Fixes #994.
2025-08-21 10:39:23 +02:00
Dustin Spicuzza 6d56ac8027 marshal: don't escape quotes unnecessarily (#991)
Only 3 consecutive quotation marks need to be quoted. We choose to quote
all quotation marks in a sequence if there are 3 or more consecutive
present.

Fixes #990

---------

Co-authored-by: Thomas Pelletier <thomas@pelletier.dev>
2025-08-21 08:19:16 +02:00
dependabot[bot] 098464b61b build(deps): bump actions/checkout from 4 to 5 (#993)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-21 08:10:55 +02:00
Oleksandr Redko 85e2448ce5 refactor: Simplify t.Fatalf (#984) 2025-05-10 15:14:34 -04:00
Thomas Pelletier ee07c9203b Update to go 1.24 (#982) 2025-04-07 07:11:38 -04:00
Alex Mikitik 014204cfb7 Replace stretchr/testify with an internal test suite (#981)
As recommended, an `internal/assert` package was added with a reduced set of assertions. All tests were then refactored to use the internal assertions. When more complex assertions were used, they have been rewritten using logic and the simplified assertions.

Fancy formatting for failures was omitted. The `internal/assert/assertions.diff` function could be overwritten for better formatting. That is where diff libraries are used in other test suites.

Refs: #872

Co-authored-by: Alex Mikitik <alex.mikitik@oracle.com>
2025-04-07 06:36:37 -04:00
Oleksandr Redko 923b2ab478 Fix typos in comments and tests (#972) 2024-11-16 11:30:13 -05:00
Thomas Pelletier af236b689f Fix goreleaser deprecated attribute name (#964)
https://goreleaser.com/deprecations/#snapshotname_template
2024-08-23 13:56:48 -04:00
Thomas Pelletier b730b2be5d Bump testing to go 1.23 (#961) 2024-08-17 16:26:05 -04:00
vito a437caafe5 Fix reflect.Pointer backward compatibility (#956) 2024-08-17 16:07:56 -04:00
guoguangwu be6c57be30 Fix readme typo(#951) 2024-08-17 15:56:40 -04:00
Daniel Weiße d55304782e Allow int, uint, and floats as map keys (#958)
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2024-08-17 15:44:21 -04:00
Daniel Weiße 0977c05dd5 Update goreleaser action to v6 and set goreleaser binary to v2 (#959)
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2024-08-17 15:40:55 -04:00
Daniel Martí bccd6e48f4 allocate unstable.Parser as part of decoder (#953)
This way, calls to Unmarshal or Decoder.Decode allocate once
at the start rather than twice.

                                │    old     │               new                │
                                │ allocs/op  │ allocs/op   vs base              │
    Unmarshal/HugoFrontMatter-8   141.0 ± 0%   140.0 ± 0%  -0.71% (p=0.002 n=6)
2024-05-24 14:49:06 -04:00
Daniel Martí 9b890cf9c5 go.mod: bump minimum and language to 1.21 (#949)
* go.mod: bump minimum and language to 1.21

CI only tests Go 1.21 and 1.22, and older versions of Go are no longer
getting any bug or security fixes, so advertise that we only support
Go 1.21 or later via go.mod.

While here, ensure the module is tidy and resolve deprecation warnings,
and remove now-unnecessary Go version build tags.

* replace sort.Slice with slices.SortFunc

The latter is more efficient, and allocates less, since sort.Slice
needs to go through sort.Interface which causes allocations.

    goos: linux
    goarch: amd64
    pkg: github.com/pelletier/go-toml/v2/benchmark
    cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
                              │     old     │                new                 │
                              │   sec/op    │   sec/op     vs base               │
    Marshal/HugoFrontMatter-8   7.612µ ± 1%   6.730µ ± 1%  -11.59% (p=0.002 n=6)

                              │     old      │                 new                 │
                              │     B/s      │     B/s       vs base               │
    Marshal/HugoFrontMatter-8   65.52Mi ± 1%   74.11Mi ± 1%  +13.11% (p=0.002 n=6)

                              │     old      │                new                 │
                              │     B/op     │     B/op      vs base              │
    Marshal/HugoFrontMatter-8   5.672Ki ± 0%   5.266Ki ± 0%  -7.16% (p=0.002 n=6)

                              │    old     │                new                │
                              │ allocs/op  │ allocs/op   vs base               │
    Marshal/HugoFrontMatter-8   85.00 ± 0%   73.00 ± 0%  -14.12% (p=0.002 n=6)
2024-05-24 10:58:39 -04:00
大可 a3d5a0bb53 fix: sync pool race condition (#947) 2024-04-29 06:02:54 -04:00
Daniel Weiße d00d2cca6e Fix indentation of custom type arrays (#944)
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2024-04-12 10:42:12 -04:00
dependabot[bot] 86608d7fca build(deps): bump github/codeql-action from 2 to 3 (#919)
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 to 3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-19 13:24:46 -04:00
dependabot[bot] 4a1877957a build(deps): bump actions/setup-go from 4 to 5 (#916)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 4 to 5.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-19 13:24:37 -04:00
dependabot[bot] 3021d6d033 build(deps): bump actions/upload-artifact from 3 to 4 (#920)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3 to 4.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-19 13:24:24 -04:00
Thomas Pelletier 32788f26f8 Update release instructions (#941) 2024-03-19 12:47:39 -04:00
rszyma 8ed6d131eb Decode: unstable/Unmarshal interface (#940)
Co-authored-by: Pavlos Karakalidis <pkarakal@pkarakal.com>
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes>
2024-03-19 12:33:12 -04:00
dependabot[bot] 7dad87762a build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#936)
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.4 to 1.9.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.4...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-12 09:06:45 -04:00
Thomas Pelletier 2b69615b5d Go 1.22 support (#935) 2024-02-27 15:30:13 -05:00
Thomas Pelletier 06fb30bf2e Decode: fix reuse of slice for array tables (#934)
When decoding into a non-empty slice, it needs to be emptied so that only the
tables contained in the document are present in the resulting value.

Arrays are not impacted because their unmarshal offset is tracked separately.

Fixes #931
2024-02-27 15:28:49 -05:00
Thomas Pelletier 2e087bdf5f Run tests on macos/M1 (#929)
https://github.blog/changelog/2024-01-30-github-actions-introducing-the-new-m1-macos-runner-available-to-open-source/
2024-01-30 19:15:50 -05:00
Thomas Pelletier caeb9f9631 Fix marshaler typos (#927) 2024-01-30 19:01:55 -05:00
Rdbo e7223fb40e fix: odd indentation in README (#928) 2024-01-30 19:01:43 -05:00
Jakub Wilk 05bedf36d8 Fix README typo (#925) 2024-01-25 15:21:33 -08:00
Daniel Graña f5486d590f Support encoding json.Number type (#923)
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes>
2024-01-25 15:21:02 -08:00
Daniel Graña 2ca21fb7b4 Support encoding of pointers to embedded structs (#924) 2024-01-23 13:06:33 -05:00
Thomas Pelletier 34765b4a9e Fix unmarshaling of nested non-exported struct (#917)
Fixes #915
2023-12-11 14:17:49 -05:00
Moritz Poldrack 358c8d2c23 Use toml-test to generate tests (#911)
Fixes: #909
2023-10-26 12:05:02 -06:00
Martin Tournoij fd8d0bf4d9 Add cmd/gotoml-test-encoder (#907) 2023-10-23 14:40:44 -06:00
Thomas Pelletier a76e18e8c5 Fix benchmark script (#905) 2023-10-02 13:49:01 -04:00
dependabot[bot] dff0c128d0 build(deps): bump docker/login-action from 2 to 3 (#901)
Bumps [docker/login-action](https://github.com/docker/login-action) from 2 to 3.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-17 18:11:24 -04:00
Thomas Pelletier 3573ce3770 Update SECURITY.md
Remove placeholder.
2023-09-04 09:43:32 -04:00
dependabot[bot] ae933f2e2a build(deps): bump actions/checkout from 3 to 4 (#896)
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-04 09:42:37 -04:00
Thomas Pelletier 3175efb395 encode: fix commented table with comment (#894)
And updated README.
2023-08-29 08:46:34 -04:00
Thomas Pelletier 9dd7f1af78 encoder: add back the commented option (#893) 2023-08-28 14:29:26 -04:00
Mikhail f. Shiryaev 4a5c27c299 decode: fix wrong indention for tables' comments (#892) 2023-08-28 13:23:11 -04:00
Thomas Pelletier 76cc96f6d8 Update LICENSE 2023-08-28 11:22:32 -04:00
Thomas Pelletier 4835627845 Decode: improve errors on integers and strings (#891) 2023-08-28 11:17:48 -04:00
Thomas Pelletier cef80b96a4 parser: add raw to integers (#890) 2023-08-28 11:02:16 -04:00
Thomas Pelletier 4040373cfd Encode: fix ignored indent of array tables (#889)
Fixes #888
2023-08-28 09:52:11 -04:00
Haiyang Wang bb026cae89 Decode: fix panic when parsing '0' as a float (#887)
Fixes #886
2023-08-22 18:07:39 +02:00
michalbiesek f7d9b9ba53 Build riscv64 binaries for linux (#884)
Signed-off-by: Michal Biesek <michalbiesek@gmail.com>
2023-08-20 19:48:24 +02:00
michalbiesek fac33d6fa8 Add support for Go 1.21 (#885)
* Add support for `Go 1.21`

Signed-off-by: Michal Biesek <michalbiesek@gmail.com>

* add go1.21 guard to fuzz_test.go

* ci: only build last two go versions

* fix workflow yaml syntax error

---------

Signed-off-by: Michal Biesek <michalbiesek@gmail.com>
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes>
2023-08-20 19:47:04 +02:00
David Barroso e183db7e69 Decode: assigned empty struct to empty defined sections (#879)
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes>
2023-07-12 10:53:17 -04:00
dependabot[bot] 60e4af8cca build(deps): bump github.com/stretchr/testify from 1.8.3 to 1.8.4 (#877)
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.3 to 1.8.4.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.3...v1.8.4)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-30 09:12:07 -04:00
MrJetBOX 8bb1e08dc7 Encode: fix support for arrays (#876) 2023-05-29 09:41:33 -04:00
Thomas Pelletier 7b980e792b Use PtrTo to not require Go 1.18 (#874) 2023-05-23 18:22:22 -04:00
dependabot[bot] 44c1513ccd build(deps): bump github.com/stretchr/testify from 1.8.2 to 1.8.3 (#871)
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.2 to 1.8.3.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.2...v1.8.3)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-19 10:46:40 -04:00
Thomas Pelletier fcf9d37d0c Comments support in unstable/Parser (#860) 2023-05-19 10:44:02 -04:00
Thomas Pelletier 986afffb7c Decode: fix decode into unsettable structs (#868)
Fixes #866
2023-05-16 09:29:50 -04:00
Thomas Pelletier 8c2c9cc986 Add example on how to use TextUnmarshaler (#867)
Ref #865
2023-05-16 08:17:21 -04:00
manunio 55ca4e35e4 fuzz: improve target perf (#864)
This attempts to improve issue types like timeout, slow_unit
and speed, as seen in the following oss-fuzz performance report:
https://oss-fuzz.com/performance-report/libFuzzer_go-toml_fuzz_toml/libfuzzer_asan_go-toml/2023-05-11
2023-05-12 16:21:17 +02:00
Gordon d34104d493 Support text Un/Marshaller for map keys (#863) 2023-05-09 17:56:57 +02:00
manunio 2aa08368fa fuzz: move fuzz_target from oss-fuzz (#861) 2023-04-28 21:38:13 -04:00
dependabot[bot] 654811fbc3 build(deps): bump actions/setup-go from 3 to 4 (#859)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-21 13:35:40 -04:00
dependabot[bot] 5c05d4d863 build(deps): bump github.com/stretchr/testify from 1.8.1 to 1.8.2 (#852)
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.1...v1.8.2)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-20 13:57:54 -04:00
59 changed files with 5831 additions and 1461 deletions
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
dry-run: false
language: go
- name: Upload Crash
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v6
if: failure() && steps.build.outcome == 'success'
with:
name: artifacts
+5 -5
View File
@@ -35,11 +35,11 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v6
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
uses: github/codeql-action/init@v4
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -47,10 +47,10 @@ jobs:
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2
uses: github/codeql-action/autobuild@v4
# ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
@@ -64,4 +64,4 @@ jobs:
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
uses: github/codeql-action/analyze@v4
+3 -3
View File
@@ -9,12 +9,12 @@ jobs:
runs-on: "ubuntu-latest"
name: report
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Setup go
uses: actions/setup-go@v3
uses: actions/setup-go@v6
with:
go-version: "1.20"
go-version: "1.24"
- name: Run tests with coverage
run: ./ci.sh coverage -d "${GITHUB_BASE_REF-HEAD}"
+36
View File
@@ -0,0 +1,36 @@
name: Go Versions Compatibility Test
on:
workflow_dispatch:
inputs:
go_versions:
description: 'Go versions to test (space-separated, e.g., "1.21 1.22 1.23")'
required: false
default: ''
type: string
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Run Go versions compatibility test
run: |
VERSIONS="${{ github.event.inputs.go_versions }}"
./test-go-versions.sh --output ./test-results $VERSIONS
- name: Upload test results
uses: actions/upload-artifact@v6
with:
name: go-versions-test-results
path: |
test-results/
retention-days: 30
+7 -7
View File
@@ -16,24 +16,24 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v3
uses: actions/setup-go@v6
with:
go-version: "1.20"
go-version: "1.24"
- name: Login to GitHub Container Registry
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v3
uses: goreleaser/goreleaser-action@v6
with:
distribution: goreleaser
version: latest
args: release ${{ inputs.args }} --rm-dist
version: '~> v2'
args: release ${{ inputs.args }} --clean
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+5 -5
View File
@@ -11,22 +11,22 @@ jobs:
build:
strategy:
matrix:
os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest']
go: [ '1.19', '1.20' ]
os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest', 'macos-14' ]
go: [ '1.23', '1.24' ]
runs-on: ${{ matrix.os }}
name: ${{ matrix.go }}/${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Setup go ${{ matrix.go }}
uses: actions/setup-go@v3
uses: actions/setup-go@v6
with:
go-version: ${{ matrix.go }}
- name: Run unit tests
run: go test -race ./...
release-check:
if: ${{ github.ref != 'refs/heads/v2' }}
uses: pelletier/go-toml/.github/workflows/release.yml@v2
uses: ./.github/workflows/release.yml
with:
args: --snapshot
+3 -1
View File
@@ -3,4 +3,6 @@ fuzz/
cmd/tomll/tomll
cmd/tomljson/tomljson
cmd/tomltestgen/tomltestgen
dist
dist
tests/
test-results
+5 -1
View File
@@ -1,3 +1,4 @@
version: 2
before:
hooks:
- go mod tidy
@@ -18,6 +19,7 @@ builds:
- linux_amd64
- linux_arm64
- linux_arm
- linux_riscv64
- windows_amd64
- windows_arm64
- windows_arm
@@ -37,6 +39,7 @@ builds:
- linux_amd64
- linux_arm64
- linux_arm
- linux_riscv64
- windows_amd64
- windows_arm64
- windows_arm
@@ -55,6 +58,7 @@ builds:
targets:
- linux_amd64
- linux_arm64
- linux_riscv64
- linux_arm
- windows_amd64
- windows_arm64
@@ -109,7 +113,7 @@ dockers:
checksum:
name_template: 'sha256sums.txt'
snapshot:
name_template: "{{ incpatch .Version }}-next"
version_template: "{{ incpatch .Version }}-next"
release:
github:
owner: pelletier
+57
View File
@@ -0,0 +1,57 @@
# Agent Guidelines for go-toml
This file provides guidelines for AI agents contributing to go-toml. All agents must follow these rules derived from [CONTRIBUTING.md](./CONTRIBUTING.md).
## Project Overview
go-toml is a TOML library for Go. The goal is to provide an easy-to-use and efficient TOML implementation that gets the job done without getting in the way.
## Code Change Rules
### Backward Compatibility
- **No backward-incompatible changes** unless explicitly discussed and approved
- Avoid breaking people's programs unless absolutely necessary
### Testing Requirements
- **All bug fixes must include regression tests**
- **All new code must be tested**
- Run tests before submitting: `go test -race ./...`
- Test coverage must not decrease. Check with:
```bash
go test -covermode=atomic -coverprofile=coverage.out
go tool cover -func=coverage.out
```
- All lines of code touched by changes should be covered by tests
### Performance Requirements
- go-toml aims to stay efficient; avoid performance regressions
- Run benchmarks to verify: `go test ./... -bench=. -count=10`
- Compare results using [benchstat](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat)
### Documentation
- New features or feature extensions must include documentation
- Documentation lives in [README.md](./README.md) and throughout source code
### Code Style
- Follow existing code format and structure
- Code must pass `go fmt`
### Commit Messages
- Commit messages must explain **why** the change is needed
- Keep messages clear and informative even if details are in the PR description
## Pull Request Checklist
Before submitting:
1. Tests pass (`go test -race ./...`)
2. No backward-incompatible changes (unless discussed)
3. Relevant documentation added/updated
4. No performance regression (verify with benchmarks)
5. Title is clear and understandable for changelog
+60 -21
View File
@@ -33,7 +33,7 @@ The documentation is present in the [README][readme] and thorough the source
code. On release, it gets updated on [pkg.go.dev][pkg.go.dev]. To make a change
to the documentation, create a pull request with your proposed changes. For
simple changes like that, the easiest way to go is probably the "Fork this
project and edit the file" button on Github, displayed at the top right of the
project and edit the file" button on GitHub, displayed at the top right of the
file. Unless it's a trivial change (for example a typo), provide a little bit of
context in your pull request description or commit message.
@@ -92,6 +92,48 @@ However, given GitHub's new policy to _not_ run Actions on pull requests until a
maintainer clicks on button, it is highly recommended that you run them locally
as you make changes.
### Test across Go versions
The repository includes tooling to test go-toml across multiple Go versions
(1.11 through 1.25) both locally and in GitHub Actions.
#### Local testing with Docker
Prerequisites: Docker installed and running, Bash shell, `rsync` command.
```bash
# Test all Go versions in parallel (default)
./test-go-versions.sh
# Test specific versions
./test-go-versions.sh 1.21 1.22 1.23
# Test sequentially (slower but uses less resources)
./test-go-versions.sh --sequential
# Verbose output with custom results directory
./test-go-versions.sh --verbose --output ./my-results 1.24 1.25
# Show all options
./test-go-versions.sh --help
```
The script creates Docker containers for each Go version and runs the full test
suite. Results are saved to a `test-results/` directory with individual logs and
a comprehensive summary report.
The script only exits with a non-zero status code if either of the two most
recent Go versions fail.
#### GitHub Actions testing (maintainers)
1. Go to the **Actions** tab in the GitHub repository
2. Select **"Go Versions Compatibility Test"** from the workflow list
3. Click **"Run workflow"**
4. Optionally customize:
- **Go versions**: Space-separated list (e.g., `1.21 1.22 1.23`)
- **Execution mode**: Parallel (faster) or sequential (more stable)
### Check coverage
We use `go tool cover` to compute test coverage. Most code editors have a way to
@@ -111,7 +153,7 @@ code lowers the coverage.
Go-toml aims to stay efficient. We rely on a set of scenarios executed with Go's
builtin benchmark systems. Because of their noisy nature, containers provided by
Github Actions cannot be reliably used for benchmarking. As a result, you are
GitHub Actions cannot be reliably used for benchmarking. As a result, you are
responsible for checking that your changes do not incur a performance penalty.
You can run their following to execute benchmarks:
@@ -165,25 +207,22 @@ Checklist:
### New release
1. Decide on the next version number. Use semver.
2. Generate release notes using [`gh`][gh]. Example:
```
$ gh api -X POST \
-F tag_name='v2.0.0-beta.5' \
-F target_commitish='v2' \
-F previous_tag_name='v2.0.0-beta.4' \
--jq '.body' \
repos/pelletier/go-toml/releases/generate-notes
```
3. Look for "Other changes". That would indicate a pull request not labeled
properly. Tweak labels and pull request titles until changelog looks good for
users.
4. [Draft new release][new-release].
5. Fill tag and target with the same value used to generate the changelog.
6. Set title to the new tag value.
7. Paste the generated changelog.
8. Check "create discussion", in the "Releases" category.
9. Check pre-release if new version is an alpha or beta.
1. Decide on the next version number. Use semver. Review commits since last
version to assess.
2. Tag release. For example:
```
git checkout v2
git pull
git tag v2.2.0
git push --tags
```
3. CI automatically builds a draft GitHub release. Review it and edit as
necessary. Look for "Other changes". That would indicate a pull request not
labeled properly. Tweak labels and pull request titles until changelog looks
good for users.
4. Check "create discussion" box, in the "Releases" category.
5. If new version is an alpha or beta only, check pre-release box.
[issues-tracker]: https://github.com/pelletier/go-toml/issues
[bug-report]: https://github.com/pelletier/go-toml/issues/new?template=bug_report.md
+2 -1
View File
@@ -1,6 +1,7 @@
The MIT License (MIT)
Copyright (c) 2013 - 2022 Thomas Pelletier, Eric Anderton
go-toml v2
Copyright (c) 2021 - 2023 Thomas Pelletier
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+148 -75
View File
@@ -45,16 +45,15 @@ to check for typos. [See example in the documentation][strict].
### Contextualized errors
When most decoding errors occur, go-toml returns [`DecodeError`][decode-err]),
When most decoding errors occur, go-toml returns [`DecodeError`][decode-err],
which contains a human readable contextualized version of the error. For
example:
```
2| key1 = "value1"
3| key2 = "missing2"
| ~~~~ missing field
4| key3 = "missing3"
5| key4 = "value4"
1| [server]
2| path = 100
| ~~~ cannot decode TOML integer into struct field toml_test.Server.Path of type string
3| port = 50
```
[decode-err]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#DecodeError
@@ -73,22 +72,46 @@ representation.
[tlt]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#LocalTime
[tldt]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#LocalDateTime
### Commented config
Since TOML is often used for configuration files, go-toml can emit documents
annotated with [comments and commented-out values][comments-example]. For
example, it can generate the following file:
```toml
# Host IP to connect to.
host = '127.0.0.1'
# Port of the remote server.
port = 4242
# Encryption parameters (optional)
# [TLS]
# cipher = 'AEAD-AES128-GCM-SHA256'
# version = 'TLS 1.3'
```
[comments-example]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#example-Marshal-Commented
## Getting started
Given the following struct, let's see how to read it and write it as TOML:
```go
type MyConfig struct {
Version int
Name string
Tags []string
Version int
Name string
Tags []string
}
```
### Unmarshaling
[`Unmarshal`][unmarshal] reads a TOML document and fills a Go structure with its
content. For example:
content.
Note that the struct variable names are _capitalized_, while the variables in the toml document are _lowercase_.
For example:
```go
doc := `
@@ -100,7 +123,7 @@ tags = ["go", "toml"]
var cfg MyConfig
err := toml.Unmarshal([]byte(doc), &cfg)
if err != nil {
panic(err)
panic(err)
}
fmt.Println("version:", cfg.Version)
fmt.Println("name:", cfg.Name)
@@ -114,6 +137,62 @@ fmt.Println("tags:", cfg.Tags)
[unmarshal]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#Unmarshal
Here is an example using tables with some simple nesting:
```go
doc := `
age = 45
fruits = ["apple", "pear"]
# these are very important!
[my-variables]
first = 1
second = 0.2
third = "abc"
# this is not so important.
[my-variables.b]
bfirst = 123
`
var Document struct {
Age int
Fruits []string
Myvariables struct {
First int
Second float64
Third string
B struct {
Bfirst int
}
} `toml:"my-variables"`
}
err := toml.Unmarshal([]byte(doc), &Document)
if err != nil {
panic(err)
}
fmt.Println("age:", Document.Age)
fmt.Println("fruits:", Document.Fruits)
fmt.Println("my-variables.first:", Document.Myvariables.First)
fmt.Println("my-variables.second:", Document.Myvariables.Second)
fmt.Println("my-variables.third:", Document.Myvariables.Third)
fmt.Println("my-variables.B.Bfirst:", Document.Myvariables.B.Bfirst)
// Output:
// age: 45
// fruits: [apple pear]
// my-variables.first: 1
// my-variables.second: 0.2
// my-variables.third: abc
// my-variables.B.Bfirst: 123
```
### Marshaling
[`Marshal`][marshal] is the opposite of Unmarshal: it represents a Go structure
@@ -121,14 +200,14 @@ as a TOML document:
```go
cfg := MyConfig{
Version: 2,
Name: "go-toml",
Tags: []string{"go", "toml"},
Version: 2,
Name: "go-toml",
Tags: []string{"go", "toml"},
}
b, err := toml.Marshal(cfg)
if err != nil {
panic(err)
panic(err)
}
fmt.Println(string(b))
@@ -156,17 +235,17 @@ the AST level. See https://pkg.go.dev/github.com/pelletier/go-toml/v2/unstable.
Execution time speedup compared to other Go TOML libraries:
<table>
<thead>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead>
<tbody>
<tr><td>Marshal/HugoFrontMatter-2</td><td>1.9x</td><td>1.9x</td></tr>
<tr><td>Marshal/ReferenceFile/map-2</td><td>1.7x</td><td>1.8x</td></tr>
<tr><td>Marshal/ReferenceFile/struct-2</td><td>2.2x</td><td>2.5x</td></tr>
<tr><td>Unmarshal/HugoFrontMatter-2</td><td>2.9x</td><td>2.9x</td></tr>
<tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.6x</td><td>2.9x</td></tr>
<tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.4x</td><td>5.3x</td></tr>
</tbody>
<thead>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead>
<tbody>
<tr><td>Marshal/HugoFrontMatter-2</td><td>1.9x</td><td>2.2x</td></tr>
<tr><td>Marshal/ReferenceFile/map-2</td><td>1.7x</td><td>2.1x</td></tr>
<tr><td>Marshal/ReferenceFile/struct-2</td><td>2.2x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/HugoFrontMatter-2</td><td>2.9x</td><td>2.7x</td></tr>
<tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.6x</td><td>2.7x</td></tr>
<tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.6x</td><td>5.1x</td></tr>
</tbody>
</table>
<details><summary>See more</summary>
<p>The table above has the results of the most common use-cases. The table below
@@ -174,22 +253,22 @@ contains the results of all benchmarks, including unrealistic ones. It is
provided for completeness.</p>
<table>
<thead>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead>
<tbody>
<tr><td>Marshal/SimpleDocument/map-2</td><td>1.8x</td><td>2.9x</td></tr>
<tr><td>Marshal/SimpleDocument/struct-2</td><td>2.7x</td><td>4.2x</td></tr>
<tr><td>Unmarshal/SimpleDocument/map-2</td><td>4.5x</td><td>3.1x</td></tr>
<tr><td>Unmarshal/SimpleDocument/struct-2</td><td>6.2x</td><td>3.9x</td></tr>
<tr><td>UnmarshalDataset/example-2</td><td>3.1x</td><td>3.5x</td></tr>
<tr><td>UnmarshalDataset/code-2</td><td>2.3x</td><td>3.1x</td></tr>
<tr><td>UnmarshalDataset/twitter-2</td><td>2.5x</td><td>2.6x</td></tr>
<tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.1x</td><td>2.2x</td></tr>
<tr><td>UnmarshalDataset/canada-2</td><td>1.6x</td><td>1.3x</td></tr>
<tr><td>UnmarshalDataset/config-2</td><td>4.3x</td><td>3.2x</td></tr>
<tr><td>[Geo mean]</td><td>2.7x</td><td>2.8x</td></tr>
</tbody>
<thead>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead>
<tbody>
<tr><td>Marshal/SimpleDocument/map-2</td><td>1.8x</td><td>2.7x</td></tr>
<tr><td>Marshal/SimpleDocument/struct-2</td><td>2.7x</td><td>3.8x</td></tr>
<tr><td>Unmarshal/SimpleDocument/map-2</td><td>3.8x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/SimpleDocument/struct-2</td><td>5.6x</td><td>4.1x</td></tr>
<tr><td>UnmarshalDataset/example-2</td><td>3.0x</td><td>3.2x</td></tr>
<tr><td>UnmarshalDataset/code-2</td><td>2.3x</td><td>2.9x</td></tr>
<tr><td>UnmarshalDataset/twitter-2</td><td>2.6x</td><td>2.7x</td></tr>
<tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.2x</td><td>2.3x</td></tr>
<tr><td>UnmarshalDataset/canada-2</td><td>1.8x</td><td>1.5x</td></tr>
<tr><td>UnmarshalDataset/config-2</td><td>4.1x</td><td>2.9x</td></tr>
<tr><td>geomean</td><td>2.7x</td><td>2.8x</td></tr>
</tbody>
</table>
<p>This table can be generated with <code>./ci.sh benchmark -a -html</code>.</p>
</details>
@@ -214,24 +293,24 @@ Go-toml provides three handy command line tools:
* `tomljson`: Reads a TOML file and outputs its JSON representation.
```
$ go install github.com/pelletier/go-toml/v2/cmd/tomljson@latest
$ tomljson --help
```
```
$ go install github.com/pelletier/go-toml/v2/cmd/tomljson@latest
$ tomljson --help
```
* `jsontoml`: Reads a JSON file and outputs a TOML representation.
```
$ go install github.com/pelletier/go-toml/v2/cmd/jsontoml@latest
$ jsontoml --help
```
```
$ go install github.com/pelletier/go-toml/v2/cmd/jsontoml@latest
$ jsontoml --help
```
* `tomll`: Lints and reformats a TOML file.
```
$ go install github.com/pelletier/go-toml/v2/cmd/tomll@latest
$ tomll --help
```
```
$ go install github.com/pelletier/go-toml/v2/cmd/tomll@latest
$ tomll --help
```
### Docker image
@@ -242,7 +321,7 @@ Those tools are also available as a [Docker image][docker]. For example, to use
docker run -i ghcr.io/pelletier/go-toml:v2 tomljson < example.toml
```
Multiple versions are availble on [ghcr.io][docker].
Multiple versions are available on [ghcr.io][docker].
[docker]: https://github.com/pelletier/go-toml/pkgs/container/go-toml
@@ -274,16 +353,16 @@ element in the interface to decode the object. For example:
```go
type inner struct {
B interface{}
B interface{}
}
type doc struct {
A interface{}
A interface{}
}
d := doc{
A: inner{
B: "Before",
},
A: inner{
B: "Before",
},
}
data := `
@@ -322,7 +401,7 @@ contained in the doc is superior to the capacity of the array. For example:
```go
type doc struct {
A [2]string
A [2]string
}
d := doc{}
err := toml.Unmarshal([]byte(`A = ["one", "two", "many"]`), &d)
@@ -497,27 +576,20 @@ is not necessary anymore.
V1 used to provide multiple struct tags: `comment`, `commented`, `multiline`,
`toml`, and `omitempty`. To behave more like the standard library, v2 has merged
`toml`, `multiline`, and `omitempty`. For example:
`toml`, `multiline`, `commented`, and `omitempty`. For example:
```go
type doc struct {
// v1
F string `toml:"field" multiline:"true" omitempty:"true"`
F string `toml:"field" multiline:"true" omitempty:"true" commented:"true"`
// v2
F string `toml:"field,multiline,omitempty"`
F string `toml:"field,multiline,omitempty,commented"`
}
```
Has a result, the `Encoder.SetTag*` methods have been removed, as there is just
one tag now.
#### `commented` tag has been removed
There is no replacement for the `commented` tag. This feature would be better
suited in a proper document model for go-toml v2, which has been [cut from
scope][nodoc] at the moment.
#### `Encoder.ArraysWithOneElementPerLine` has been renamed
The new name is `Encoder.SetArraysMultiline`. The behavior should be the same.
@@ -553,10 +625,11 @@ complete solutions exist out there.
## Versioning
Go-toml follows [Semantic Versioning](https://semver.org). The supported version
of [TOML](https://github.com/toml-lang/toml) is indicated at the beginning of
this document. The last two major versions of Go are supported
(see [Go Release Policy](https://golang.org/doc/devel/release.html#policy)).
Expect for parts explicitly marked otherwise, go-toml follows [Semantic
Versioning](https://semver.org). The supported version of
[TOML](https://github.com/toml-lang/toml) is indicated at the beginning of this
document. The last two major versions of Go are supported (see [Go Release
Policy](https://golang.org/doc/devel/release.html#policy)).
## License
-3
View File
@@ -2,9 +2,6 @@
## Supported Versions
Use this section to tell people about which versions of your project are
currently being supported with security updates.
| Version | Supported |
| ---------- | ------------------ |
| Latest 2.x | :white_check_mark: |
+10 -10
View File
@@ -3,13 +3,13 @@ package benchmark_test
import (
"compress/gzip"
"encoding/json"
"io/ioutil"
"io"
"os"
"path/filepath"
"testing"
"github.com/pelletier/go-toml/v2"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
var bench_inputs = []struct {
@@ -35,11 +35,11 @@ func TestUnmarshalDatasetCode(t *testing.T) {
buf := fixture(t, tc.name)
var v interface{}
require.NoError(t, toml.Unmarshal(buf, &v))
assert.NoError(t, toml.Unmarshal(buf, &v))
b, err := json.Marshal(v)
require.NoError(t, err)
require.Equal(t, len(b), tc.jsonLen)
assert.NoError(t, err)
assert.Equal(t, len(b), tc.jsonLen)
})
}
}
@@ -53,7 +53,7 @@ func BenchmarkUnmarshalDataset(b *testing.B) {
b.ResetTimer()
for i := 0; i < b.N; i++ {
var v interface{}
require.NoError(b, toml.Unmarshal(buf, &v))
assert.NoError(b, toml.Unmarshal(buf, &v))
}
})
}
@@ -68,13 +68,13 @@ func fixture(tb testing.TB, path string) []byte {
if os.IsNotExist(err) {
tb.Skip("benchmark fixture not found:", file)
}
require.NoError(tb, err)
assert.NoError(tb, err)
defer f.Close()
gz, err := gzip.NewReader(f)
require.NoError(tb, err)
assert.NoError(tb, err)
buf, err := ioutil.ReadAll(gz)
require.NoError(tb, err)
buf, err := io.ReadAll(gz)
assert.NoError(tb, err)
return buf
}
+8 -8
View File
@@ -2,12 +2,12 @@ package benchmark_test
import (
"bytes"
"io/ioutil"
"os"
"testing"
"time"
"github.com/pelletier/go-toml/v2"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
func TestUnmarshalSimple(t *testing.T) {
@@ -59,7 +59,7 @@ func BenchmarkUnmarshal(b *testing.B) {
})
b.Run("ReferenceFile", func(b *testing.B) {
bytes, err := ioutil.ReadFile("benchmark.toml")
bytes, err := os.ReadFile("benchmark.toml")
if err != nil {
b.Fatal(err)
}
@@ -165,7 +165,7 @@ func BenchmarkMarshal(b *testing.B) {
})
b.Run("ReferenceFile", func(b *testing.B) {
bytes, err := ioutil.ReadFile("benchmark.toml")
bytes, err := os.ReadFile("benchmark.toml")
if err != nil {
b.Fatal(err)
}
@@ -344,11 +344,11 @@ type benchmarkDoc struct {
}
func TestUnmarshalReferenceFile(t *testing.T) {
bytes, err := ioutil.ReadFile("benchmark.toml")
require.NoError(t, err)
bytes, err := os.ReadFile("benchmark.toml")
assert.NoError(t, err)
d := benchmarkDoc{}
err = toml.Unmarshal(bytes, &d)
require.NoError(t, err)
assert.NoError(t, err)
expected := benchmarkDoc{
Table: struct {
@@ -627,7 +627,7 @@ trimmed in raw strings.
},
}
require.Equal(t, expected, d)
assert.Equal(t, expected, d)
}
var hugoFrontMatterbytes = []byte(`
+14 -9
View File
@@ -77,8 +77,9 @@ cover() {
pushd "$dir"
go test -covermode=atomic -coverpkg=./... -coverprofile=coverage.out.tmp ./...
cat coverage.out.tmp | grep -v testsuite | grep -v tomltestgen | grep -v gotoml-test-decoder > coverage.out
grep -Ev '(fuzz|testsuite|tomltestgen|gotoml-test-decoder|gotoml-test-encoder)' coverage.out.tmp > coverage.out
go tool cover -func=coverage.out
echo "Coverage profile for ${branch}: ${dir}/coverage.out" >&2
popd
if [ "${branch}" != "HEAD" ]; then
@@ -151,7 +152,7 @@ bench() {
fi
export GOMAXPROCS=2
nice -n -19 taskset --cpu-list 0,1 go test '-bench=^Benchmark(Un)?[mM]arshal' -count=5 -run=Nothing ./... | tee "${out}"
go test '-bench=^Benchmark(Un)?[mM]arshal' -count=10 -run=Nothing ./... | tee "${out}"
popd
if [ "${branch}" != "HEAD" ]; then
@@ -160,10 +161,12 @@ bench() {
}
fmktemp() {
if mktemp --version|grep GNU >/dev/null; then
mktemp --suffix=-$1;
if mktemp --version &> /dev/null; then
# GNU
mktemp --suffix=-$1
else
mktemp -t $1;
# BSD
mktemp -t $1
fi
}
@@ -183,12 +186,14 @@ with open(sys.argv[1]) as f:
lines.append(line.split(','))
results = []
for line in reversed(lines[1:]):
for line in reversed(lines[2:]):
if len(line) < 8 or line[0] == "":
continue
v2 = float(line[1])
results.append([
line[0].replace("-32", ""),
"%.1fx" % (float(line[3])/v2), # v1
"%.1fx" % (float(line[5])/v2), # bs
"%.1fx" % (float(line[7])/v2), # bs
])
# move geomean to the end
results.append(results[0])
@@ -259,10 +264,10 @@ benchmark() {
if [ "$1" = "-html" ]; then
tmpcsv=`fmktemp csv`
benchstat -csv -geomean go-toml-v2.txt go-toml-v1.txt bs-toml.txt > $tmpcsv
benchstat -format csv go-toml-v2.txt go-toml-v1.txt bs-toml.txt > $tmpcsv
benchstathtml $tmpcsv
else
benchstat -geomean go-toml-v2.txt go-toml-v1.txt bs-toml.txt
benchstat go-toml-v2.txt go-toml-v1.txt bs-toml.txt
fi
rm -f go-toml-v2.txt go-toml-v1.txt bs-toml.txt
@@ -0,0 +1,30 @@
package main
import (
"flag"
"log"
"os"
"path"
"github.com/pelletier/go-toml/v2/internal/testsuite"
)
func main() {
log.SetFlags(0)
flag.Usage = usage
flag.Parse()
if flag.NArg() != 0 {
flag.Usage()
}
err := testsuite.EncodeStdin()
if err != nil {
log.Fatal(err)
}
}
func usage() {
log.Printf("Usage: %s < toml-file\n", path.Base(os.Args[0]))
flag.PrintDefaults()
os.Exit(1)
}
+12 -1
View File
@@ -19,6 +19,7 @@ package main
import (
"encoding/json"
"flag"
"io"
"github.com/pelletier/go-toml/v2"
@@ -33,7 +34,11 @@ Reading from a file:
jsontoml file.json > file.toml
`
var useJsonNumber bool
func main() {
flag.BoolVar(&useJsonNumber, "use-json-number", false, "unmarshal numbers into `json.Number` type instead of as `float64`")
p := cli.Program{
Usage: usage,
Fn: convert,
@@ -45,11 +50,17 @@ func convert(r io.Reader, w io.Writer) error {
var v interface{}
d := json.NewDecoder(r)
e := toml.NewEncoder(w)
if useJsonNumber {
d.UseNumber()
e.SetMarshalJsonNumbers(true)
}
err := d.Decode(&v)
if err != nil {
return err
}
e := toml.NewEncoder(w)
return e.Encode(v)
}
+21 -7
View File
@@ -5,16 +5,16 @@ import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
func TestConvert(t *testing.T) {
examples := []struct {
name string
input string
expected string
errors bool
name string
input string
expected string
errors bool
useJsonNumber bool
}{
{
name: "valid json",
@@ -26,6 +26,19 @@ func TestConvert(t *testing.T) {
}`,
expected: `[mytoml]
a = 42.0
`,
},
{
name: "use json number",
useJsonNumber: true,
input: `
{
"mytoml": {
"a": 42
}
}`,
expected: `[mytoml]
a = 42
`,
},
{
@@ -37,9 +50,10 @@ a = 42.0
for _, e := range examples {
b := new(bytes.Buffer)
useJsonNumber = e.useJsonNumber
err := convert(strings.NewReader(e.input), b)
if e.errors {
require.Error(t, err)
assert.Error(t, err)
} else {
assert.NoError(t, err)
assert.Equal(t, e.expected, b.String())
+2 -3
View File
@@ -7,8 +7,7 @@ import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
func TestConvert(t *testing.T) {
@@ -46,7 +45,7 @@ a = 42`),
b := new(bytes.Buffer)
err := convert(e.input, b)
if e.errors {
require.Error(t, err)
assert.Error(t, err)
} else {
assert.NoError(t, err)
assert.Equal(t, e.expected, b.String())
+2 -3
View File
@@ -5,8 +5,7 @@ import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
func TestConvert(t *testing.T) {
@@ -36,7 +35,7 @@ a = 42.0
b := new(bytes.Buffer)
err := convert(strings.NewReader(e.input), b)
if e.errors {
require.Error(t, err)
assert.Error(t, err)
} else {
assert.NoError(t, err)
assert.Equal(t, e.expected, b.String())
+45 -88
View File
@@ -7,17 +7,13 @@
package main
import (
"archive/zip"
"bytes"
"flag"
"fmt"
"go/format"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"regexp"
"path/filepath"
"strconv"
"strings"
"text/template"
@@ -64,30 +60,6 @@ const srcTemplate = "// Generated by tomltestgen for toml-test ref {{.Ref}} on {
"}\n" +
"{{end}}\n"
func downloadTmpFile(url string) string {
log.Println("starting to download file from", url)
resp, err := http.Get(url)
if err != nil {
panic(err)
}
defer resp.Body.Close()
tmpfile, err := ioutil.TempFile("", "toml-test-*.zip")
if err != nil {
panic(err)
}
defer tmpfile.Close()
copiedLen, err := io.Copy(tmpfile, resp.Body)
if err != nil {
panic(err)
}
if resp.ContentLength > 0 && copiedLen != resp.ContentLength {
panic(fmt.Errorf("copied %d bytes, request body had %d", copiedLen, resp.ContentLength))
}
return tmpfile.Name()
}
func kebabToCamel(kebab string) string {
camel := ""
nextUpper := true
@@ -107,19 +79,6 @@ func kebabToCamel(kebab string) string {
return camel
}
func readFileFromZip(f *zip.File) string {
reader, err := f.Open()
if err != nil {
panic(err)
}
defer reader.Close()
bytes, err := ioutil.ReadAll(reader)
if err != nil {
panic(err)
}
return string(bytes)
}
func templateGoStr(input string) string {
return strconv.Quote(input)
}
@@ -138,61 +97,59 @@ func main() {
flag.Usage = usage
flag.Parse()
url := "https://codeload.github.com/BurntSushi/toml-test/zip/" + *ref
resultFile := downloadTmpFile(url)
defer os.Remove(resultFile)
log.Println("file written to", resultFile)
zipReader, err := zip.OpenReader(resultFile)
if err != nil {
panic(err)
}
defer zipReader.Close()
collection := testsCollection{
Ref: *ref,
Timestamp: time.Now().Format(time.RFC3339),
}
zipFilesMap := map[string]*zip.File{}
dirContent, _ := filepath.Glob("tests/invalid/**/*.toml")
for _, f := range dirContent {
filename := strings.TrimPrefix(f, "tests/valid/")
name := kebabToCamel(strings.TrimSuffix(filename, ".toml"))
name = strings.ReplaceAll(name, ".", "_")
for _, f := range zipReader.File {
zipFilesMap[f.Name] = f
log.Printf("> [%s] %s\n", "invalid", name)
tomlContent, err := os.ReadFile(f)
if err != nil {
fmt.Printf("failed to read test file: %s\n", err)
os.Exit(1)
}
collection.Invalid = append(collection.Invalid, invalid{
Name: name,
Input: string(tomlContent),
})
collection.Count++
}
testFileRegexp := regexp.MustCompile(`([^/]+/tests/(valid|invalid)/(.+))\.(toml)`)
for _, f := range zipReader.File {
groups := testFileRegexp.FindStringSubmatch(f.Name)
if len(groups) > 0 {
name := kebabToCamel(groups[3])
testType := groups[2]
dirContent, _ = filepath.Glob("tests/valid/**/*.toml")
for _, f := range dirContent {
filename := strings.TrimPrefix(f, "tests/valid/")
name := kebabToCamel(strings.TrimSuffix(filename, ".toml"))
name = strings.ReplaceAll(name, ".", "_")
log.Printf("> [%s] %s\n", testType, name)
log.Printf("> [%s] %s\n", "valid", name)
tomlContent := readFileFromZip(f)
switch testType {
case "invalid":
collection.Invalid = append(collection.Invalid, invalid{
Name: name,
Input: tomlContent,
})
collection.Count++
case "valid":
baseFilePath := groups[1]
jsonFilePath := baseFilePath + ".json"
jsonContent := readFileFromZip(zipFilesMap[jsonFilePath])
collection.Valid = append(collection.Valid, valid{
Name: name,
Input: tomlContent,
JsonRef: jsonContent,
})
collection.Count++
default:
panic(fmt.Sprintf("unknown test type: %s", testType))
}
tomlContent, err := os.ReadFile(f)
if err != nil {
fmt.Printf("failed reading test file: %s\n", err)
os.Exit(1)
}
filename = strings.TrimSuffix(f, ".toml")
jsonContent, err := os.ReadFile(filename + ".json")
if err != nil {
fmt.Printf("failed reading validation json: %s\n", err)
os.Exit(1)
}
collection.Valid = append(collection.Valid, valid{
Name: name,
Input: string(tomlContent),
JsonRef: string(jsonContent),
})
collection.Count++
}
log.Printf("Collected %d tests from toml-test\n", collection.Count)
@@ -202,7 +159,7 @@ func main() {
}
t := template.Must(template.New("src").Funcs(funcMap).Parse(srcTemplate))
buf := new(bytes.Buffer)
err = t.Execute(buf, collection)
err := t.Execute(buf, collection)
if err != nil {
panic(err)
}
@@ -216,7 +173,7 @@ func main() {
return
}
err = os.WriteFile(*out, outputBytes, 0644)
err = os.WriteFile(*out, outputBytes, 0o644)
if err != nil {
panic(err)
}
+1 -1
View File
@@ -318,7 +318,7 @@ func parseFloat(b []byte) (float64, error) {
if cleaned[0] == '+' || cleaned[0] == '-' {
start = 1
}
if cleaned[start] == '0' && isDigit(cleaned[start+1]) {
if cleaned[start] == '0' && len(cleaned) > start+1 && isDigit(cleaned[start+1]) {
return 0, unstable.NewParserError(b, "float integer part cannot have leading zeroes")
}
+13 -3
View File
@@ -5,7 +5,6 @@ import (
"strconv"
"strings"
"github.com/pelletier/go-toml/v2/internal/danger"
"github.com/pelletier/go-toml/v2/unstable"
)
@@ -54,6 +53,17 @@ func (s *StrictMissingError) String() string {
return buf.String()
}
// Unwrap returns wrapped decode errors
//
// Implements errors.Join() interface.
func (s *StrictMissingError) Unwrap() []error {
var errs []error
for i := range s.Errors {
errs = append(errs, &s.Errors[i])
}
return errs
}
type Key []string
// Error returns the error message contained in the DecodeError.
@@ -78,7 +88,7 @@ func (e *DecodeError) Key() Key {
return e.key
}
// decodeErrorFromHighlight creates a DecodeError referencing a highlighted
// wrapDecodeError creates a DecodeError referencing a highlighted
// range of bytes from document.
//
// highlight needs to be a sub-slice of document, or this function panics.
@@ -88,7 +98,7 @@ func (e *DecodeError) Key() Key {
//
//nolint:funlen
func wrapDecodeError(document []byte, de *unstable.ParserError) *DecodeError {
offset := danger.SubsliceOffset(document, de.Highlight)
offset := cap(document) - cap(de.Highlight)
errMessage := de.Error()
errLine, errColumn := positionAtEnd(document[:offset])
+16 -1
View File
@@ -7,8 +7,8 @@ import (
"strings"
"testing"
"github.com/pelletier/go-toml/v2/internal/assert"
"github.com/pelletier/go-toml/v2/unstable"
"github.com/stretchr/testify/assert"
)
//nolint:funlen
@@ -205,6 +205,21 @@ func TestDecodeError_Accessors(t *testing.T) {
assert.Equal(t, "bar", e.String())
}
func TestStrictErrorUnwrap(t *testing.T) {
fo := bytes.NewBufferString(`
Missing = 1
OtherMissing = 1
`)
var out struct{}
err := NewDecoder(fo).DisallowUnknownFields().Decode(&out)
assert.Error(t, err)
strictErr := &StrictMissingError{}
assert.True(t, errors.As(err, &strictErr))
assert.Equal(t, 2, len(strictErr.Unwrap()))
}
func ExampleDecodeError() {
doc := `name = 123__456`
+37
View File
@@ -0,0 +1,37 @@
package toml_test
import (
"fmt"
"log"
"strconv"
"github.com/pelletier/go-toml/v2"
)
type customInt int
func (i *customInt) UnmarshalText(b []byte) error {
x, err := strconv.ParseInt(string(b), 10, 32)
if err != nil {
return err
}
*i = customInt(x * 100)
return nil
}
type doc struct {
Value customInt
}
func ExampleUnmarshal_textUnmarshal() {
var x doc
data := []byte(`value = "42"`)
err := toml.Unmarshal(data, &x)
if err != nil {
log.Fatal(err)
}
fmt.Println(x)
// Output:
// {4200}
}
+15 -15
View File
@@ -4,28 +4,28 @@ import (
"testing"
"github.com/pelletier/go-toml/v2"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
func TestFastSimpleInt(t *testing.T) {
m := map[string]int64{}
err := toml.Unmarshal([]byte(`a = 42`), &m)
require.NoError(t, err)
require.Equal(t, map[string]int64{"a": 42}, m)
assert.NoError(t, err)
assert.Equal(t, map[string]int64{"a": 42}, m)
}
func TestFastSimpleFloat(t *testing.T) {
m := map[string]float64{}
err := toml.Unmarshal([]byte("a = 42\nb = 1.1\nc = 12341234123412341234123412341234"), &m)
require.NoError(t, err)
require.Equal(t, map[string]float64{"a": 42, "b": 1.1, "c": 1.2341234123412342e+31}, m)
assert.NoError(t, err)
assert.Equal(t, map[string]float64{"a": 42, "b": 1.1, "c": 1.2341234123412342e+31}, m)
}
func TestFastSimpleString(t *testing.T) {
m := map[string]string{}
err := toml.Unmarshal([]byte(`a = "hello"`), &m)
require.NoError(t, err)
require.Equal(t, map[string]string{"a": "hello"}, m)
assert.NoError(t, err)
assert.Equal(t, map[string]string{"a": "hello"}, m)
}
func TestFastSimpleInterface(t *testing.T) {
@@ -33,8 +33,8 @@ func TestFastSimpleInterface(t *testing.T) {
err := toml.Unmarshal([]byte(`
a = "hello"
b = 42`), &m)
require.NoError(t, err)
require.Equal(t, map[string]interface{}{
assert.NoError(t, err)
assert.Equal(t, map[string]interface{}{
"a": "hello",
"b": int64(42),
}, m)
@@ -46,8 +46,8 @@ func TestFastMultipartKeyInterface(t *testing.T) {
a.interim = "test"
a.b.c = "hello"
b = 42`), &m)
require.NoError(t, err)
require.Equal(t, map[string]interface{}{
assert.NoError(t, err)
assert.Equal(t, map[string]interface{}{
"a": map[string]interface{}{
"interim": "test",
"b": map[string]interface{}{
@@ -66,8 +66,8 @@ func TestFastExistingMap(t *testing.T) {
ints.one = 1
ints.two = 2
strings.yo = "hello"`), &m)
require.NoError(t, err)
require.Equal(t, map[string]interface{}{
assert.NoError(t, err)
assert.Equal(t, map[string]interface{}{
"ints": map[string]interface{}{
"one": int64(1),
"two": int64(2),
@@ -90,9 +90,9 @@ func TestFastArrayTable(t *testing.T) {
m := map[string]interface{}{}
err := toml.Unmarshal(b, &m)
require.NoError(t, err)
assert.NoError(t, err)
require.Equal(t, map[string]interface{}{
assert.Equal(t, map[string]interface{}{
"root": map[string]interface{}{
"nested": []interface{}{
map[string]interface{}{
+4 -7
View File
@@ -1,19 +1,16 @@
//go:build go1.18 || go1.19 || go1.20
// +build go1.18 go1.19 go1.20
package toml_test
import (
"io/ioutil"
"os"
"strings"
"testing"
"github.com/pelletier/go-toml/v2"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
func FuzzUnmarshal(f *testing.F) {
file, err := ioutil.ReadFile("benchmark/benchmark.toml")
file, err := os.ReadFile("benchmark/benchmark.toml")
if err != nil {
panic(err)
}
@@ -51,6 +48,6 @@ func FuzzUnmarshal(f *testing.F) {
if err != nil {
t.Fatalf("failed round trip: %s", err)
}
require.Equal(t, v, v2)
assert.Equal(t, v, v2)
})
}
+1 -3
View File
@@ -1,5 +1,3 @@
module github.com/pelletier/go-toml/v2
go 1.16
require github.com/stretchr/testify v1.8.1
go 1.21.0
-17
View File
@@ -1,17 +0,0 @@
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+135
View File
@@ -0,0 +1,135 @@
package assert
import (
"bytes"
"fmt"
"reflect"
"strings"
"testing"
)
// True asserts that an expression is true.
func True(t testing.TB, ok bool, msgAndArgs ...any) {
if ok {
return
}
t.Helper()
t.Fatal(formatMsgAndArgs("Expected expression to be true", msgAndArgs...))
}
// False asserts that an expression is false.
func False(t testing.TB, ok bool, msgAndArgs ...any) {
if !ok {
return
}
t.Helper()
t.Fatal(formatMsgAndArgs("Expected expression to be false", msgAndArgs...))
}
// Equal asserts that "expected" and "actual" are equal.
func Equal[T any](t testing.TB, expected, actual T, msgAndArgs ...any) {
if objectsAreEqual(expected, actual) {
return
}
t.Helper()
msg := formatMsgAndArgs("Expected values to be equal:", msgAndArgs...)
t.Fatalf("%s\n%s", msg, diff(expected, actual))
}
// Error asserts that an error is not nil.
func Error(t testing.TB, err error, msgAndArgs ...any) {
if err != nil {
return
}
t.Helper()
t.Fatal(formatMsgAndArgs("Expected an error", msgAndArgs...))
}
// NoError asserts that an error is nil.
func NoError(t testing.TB, err error, msgAndArgs ...any) {
if err == nil {
return
}
t.Helper()
msg := formatMsgAndArgs("Unexpected error:", msgAndArgs...)
t.Fatalf("%s\n%+v", msg, err)
}
// Panics asserts that the given function panics.
func Panics(t testing.TB, fn func(), msgAndArgs ...any) {
t.Helper()
defer func() {
if recover() == nil {
msg := formatMsgAndArgs("Expected function to panic", msgAndArgs...)
t.Fatal(msg)
}
}()
fn()
}
// Zero asserts that a value is its zero value.
func Zero[T any](t testing.TB, value T, msgAndArgs ...any) {
var zero T
if objectsAreEqual(value, zero) {
return
}
val := reflect.ValueOf(value)
if (val.Kind() == reflect.Slice || val.Kind() == reflect.Map || val.Kind() == reflect.Array) && val.Len() == 0 {
return
}
t.Helper()
msg := formatMsgAndArgs("Expected zero value but got:", msgAndArgs...)
t.Fatalf("%s\n%v", msg, value)
}
func NotZero[T any](t testing.TB, value T, msgAndArgs ...any) {
var zero T
if !objectsAreEqual(value, zero) {
val := reflect.ValueOf(value)
if !((val.Kind() == reflect.Slice || val.Kind() == reflect.Map || val.Kind() == reflect.Array) && val.Len() == 0) {
return
}
}
t.Helper()
msg := formatMsgAndArgs("Unexpected zero value:", msgAndArgs...)
t.Fatalf("%s\n%v", msg, value)
}
func formatMsgAndArgs(msg string, args ...any) string {
if len(args) == 0 {
return msg
}
format, ok := args[0].(string)
if !ok {
panic("message argument must be a fmt string")
}
return fmt.Sprintf(format, args[1:]...)
}
func diff(expected, actual any) string {
lines := []string{
"expected:",
fmt.Sprintf("%v", expected),
"actual:",
fmt.Sprintf("%v", actual),
}
return strings.Join(lines, "\n")
}
func objectsAreEqual(expected, actual any) bool {
if expected == nil || actual == nil {
return expected == actual
}
if exp, eok := expected.([]byte); eok {
if act, aok := actual.([]byte); aok {
return bytes.Equal(exp, act)
}
}
if exp, eok := expected.(string); eok {
if act, aok := actual.(string); aok {
return exp == act
}
}
return reflect.DeepEqual(expected, actual)
}
+184
View File
@@ -0,0 +1,184 @@
package assert
import (
"fmt"
"testing"
)
type Data struct {
Label string
Value int64
}
func TestBadMessage(t *testing.T) {
invalidMessage := func() { True(t, false, 1234) }
assertOk(t, "Non-fmt message value", func(t testing.TB) {
Panics(t, invalidMessage)
})
assertFail(t, "Non-fmt message value", func(t testing.TB) {
True(t, false, "example %s", "message")
})
}
func TestTrue(t *testing.T) {
assertOk(t, "Succeed", func(t testing.TB) {
True(t, 1 > 0)
})
assertFail(t, "Fail", func(t testing.TB) {
True(t, 1 < 0)
})
}
func TestFalse(t *testing.T) {
assertOk(t, "Succeed", func(t testing.TB) {
False(t, 1 < 0)
})
assertFail(t, "Fail", func(t testing.TB) {
False(t, 1 > 0)
})
}
func TestEqual(t *testing.T) {
assertOk(t, "Nil", func(t testing.TB) {
Equal(t, interface{}(nil), interface{}(nil))
})
assertOk(t, "Identical structs", func(t testing.TB) {
Equal(t, Data{"expected", 1234}, Data{"expected", 1234})
})
assertFail(t, "Different structs", func(t testing.TB) {
Equal(t, Data{"expected", 1234}, Data{"actual", 1234})
})
assertOk(t, "Identical numbers", func(t testing.TB) {
Equal(t, 1234, 1234)
})
assertFail(t, "Identical numbers", func(t testing.TB) {
Equal(t, 1234, 1324)
})
assertOk(t, "Zero-length byte arrays", func(t testing.TB) {
Equal(t, []byte(nil), []byte(""))
})
assertOk(t, "Identical byte arrays", func(t testing.TB) {
Equal(t, []byte{1, 2, 3, 4}, []byte{1, 2, 3, 4})
})
assertFail(t, "Different byte arrays", func(t testing.TB) {
Equal(t, []byte{1, 2, 3, 4}, []byte{1, 3, 2, 4})
})
assertOk(t, "Identical strings", func(t testing.TB) {
Equal(t, "example", "example")
})
assertFail(t, "Identical strings", func(t testing.TB) {
Equal(t, "example", "elpmaxe")
})
}
func TestError(t *testing.T) {
assertOk(t, "Error", func(t testing.TB) {
Error(t, fmt.Errorf("example"))
})
assertFail(t, "Nil", func(t testing.TB) {
Error(t, nil)
})
}
func TestNoError(t *testing.T) {
assertFail(t, "Error", func(t testing.TB) {
NoError(t, fmt.Errorf("example"))
})
assertOk(t, "Nil", func(t testing.TB) {
NoError(t, nil)
})
}
func TestPanics(t *testing.T) {
willPanic := func() { panic("example") }
wontPanic := func() {}
assertOk(t, "Will panic", func(t testing.TB) {
Panics(t, willPanic)
})
assertFail(t, "Won't panic", func(t testing.TB) {
Panics(t, wontPanic)
})
}
func TestZero(t *testing.T) {
assertOk(t, "Empty struct", func(t testing.TB) {
Zero(t, Data{})
})
assertFail(t, "Non-empty struct", func(t testing.TB) {
Zero(t, Data{Label: "example"})
})
assertOk(t, "Nil slice", func(t testing.TB) {
var slice []int
Zero(t, slice)
})
assertFail(t, "Non-empty slice", func(t testing.TB) {
slice := []int{1, 2, 3, 4}
Zero(t, slice)
})
assertOk(t, "Zero-length slice", func(t testing.TB) {
slice := []int{}
Zero(t, slice)
})
}
func TestNotZero(t *testing.T) {
assertFail(t, "Empty struct", func(t testing.TB) {
zero := Data{}
NotZero(t, zero)
})
assertOk(t, "Non-empty struct", func(t testing.TB) {
notZero := Data{Label: "example"}
NotZero(t, notZero)
})
assertFail(t, "Nil slice", func(t testing.TB) {
var slice []int
NotZero(t, slice)
})
assertFail(t, "Zero-length slice", func(t testing.TB) {
slice := []int{}
NotZero(t, slice)
})
assertOk(t, "Non-empty slice", func(t testing.TB) {
slice := []int{1, 2, 3, 4}
NotZero(t, slice)
})
}
type testCase struct {
*testing.T
failed string
}
func (t *testCase) Fatal(args ...interface{}) {
t.failed = fmt.Sprint(args...)
}
func (t *testCase) Fatalf(message string, args ...interface{}) {
t.failed = fmt.Sprintf(message, args...)
}
func assertFail(t *testing.T, name string, fn func(t testing.TB)) {
t.Helper()
t.Run(name, func(t *testing.T) {
t.Helper()
test := &testCase{T: t}
fn(test)
if test.failed == "" {
t.Fatal("Test expected to fail but did not")
} else {
t.Log(test.failed)
}
})
}
func assertOk(t *testing.T, name string, fn func(t testing.TB)) {
t.Helper()
t.Run(name, func(t *testing.T) {
t.Helper()
test := &testCase{T: t}
fn(test)
if test.failed != "" {
t.Fatal("Test expected to succeed but did not:\n", test.failed)
}
})
}
+3 -4
View File
@@ -6,7 +6,6 @@ import (
"flag"
"fmt"
"io"
"io/ioutil"
"os"
"github.com/pelletier/go-toml/v2"
@@ -23,7 +22,7 @@ type Program struct {
}
func (p *Program) Execute() {
flag.Usage = func() { fmt.Fprintf(os.Stderr, p.Usage) }
flag.Usage = func() { fmt.Fprint(os.Stderr, p.Usage) }
flag.Parse()
os.Exit(p.main(flag.Args(), os.Stdin, os.Stdout, os.Stderr))
}
@@ -72,7 +71,7 @@ func (p *Program) runAllFilesInPlace(files []string) error {
}
func (p *Program) runFileInPlace(path string) error {
in, err := ioutil.ReadFile(path)
in, err := os.ReadFile(path)
if err != nil {
return err
}
@@ -84,5 +83,5 @@ func (p *Program) runFileInPlace(path string) error {
return err
}
return ioutil.WriteFile(path, out.Bytes(), 0600)
return os.WriteFile(path, out.Bytes(), 0600)
}
+37 -39
View File
@@ -4,15 +4,13 @@ import (
"bytes"
"fmt"
"io"
"io/ioutil"
"os"
"path"
"strings"
"testing"
"github.com/pelletier/go-toml/v2"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
func processMain(args []string, input io.Reader, stdout, stderr io.Writer, f ConvertFn) int {
@@ -30,8 +28,8 @@ func TestProcessMainStdin(t *testing.T) {
})
assert.Equal(t, 0, exit)
assert.Empty(t, stdout.String())
assert.Empty(t, stderr.String())
assert.Zero(t, stdout.String())
assert.Zero(t, stderr.String())
}
func TestProcessMainStdinErr(t *testing.T) {
@@ -44,8 +42,8 @@ func TestProcessMainStdinErr(t *testing.T) {
})
assert.Equal(t, -1, exit)
assert.Empty(t, stdout.String())
assert.NotEmpty(t, stderr.String())
assert.Zero(t, stdout.String())
assert.NotZero(t, stderr.String())
}
func TestProcessMainStdinDecodeErr(t *testing.T) {
@@ -59,16 +57,16 @@ func TestProcessMainStdinDecodeErr(t *testing.T) {
})
assert.Equal(t, -1, exit)
assert.Empty(t, stdout.String())
assert.Contains(t, stderr.String(), "error occurred at")
assert.Zero(t, stdout.String())
assert.True(t, strings.Contains(stderr.String(), "error occurred at"))
}
func TestProcessMainFileExists(t *testing.T) {
tmpfile, err := ioutil.TempFile("", "example")
require.NoError(t, err)
tmpfile, err := os.CreateTemp("", "example")
assert.NoError(t, err)
defer os.Remove(tmpfile.Name())
_, err = tmpfile.Write([]byte(`some data`))
require.NoError(t, err)
assert.NoError(t, err)
stdout := new(bytes.Buffer)
stderr := new(bytes.Buffer)
@@ -78,8 +76,8 @@ func TestProcessMainFileExists(t *testing.T) {
})
assert.Equal(t, 0, exit)
assert.Empty(t, stdout.String())
assert.Empty(t, stderr.String())
assert.Zero(t, stdout.String())
assert.Zero(t, stderr.String())
}
func TestProcessMainFileDoesNotExist(t *testing.T) {
@@ -91,22 +89,22 @@ func TestProcessMainFileDoesNotExist(t *testing.T) {
})
assert.Equal(t, -1, exit)
assert.Empty(t, stdout.String())
assert.NotEmpty(t, stderr.String())
assert.Zero(t, stdout.String())
assert.NotZero(t, stderr.String())
}
func TestProcessMainFilesInPlace(t *testing.T) {
dir, err := ioutil.TempDir("", "")
require.NoError(t, err)
dir, err := os.MkdirTemp("", "")
assert.NoError(t, err)
defer os.RemoveAll(dir)
path1 := path.Join(dir, "file1")
path2 := path.Join(dir, "file2")
err = ioutil.WriteFile(path1, []byte("content 1"), 0600)
require.NoError(t, err)
err = ioutil.WriteFile(path2, []byte("content 2"), 0600)
require.NoError(t, err)
err = os.WriteFile(path1, []byte("content 1"), 0600)
assert.NoError(t, err)
err = os.WriteFile(path2, []byte("content 2"), 0600)
assert.NoError(t, err)
p := Program{
Fn: dummyFileFn,
@@ -115,15 +113,15 @@ func TestProcessMainFilesInPlace(t *testing.T) {
exit := p.main([]string{path1, path2}, os.Stdin, os.Stdout, os.Stderr)
require.Equal(t, 0, exit)
assert.Equal(t, 0, exit)
v1, err := ioutil.ReadFile(path1)
require.NoError(t, err)
require.Equal(t, "1", string(v1))
v1, err := os.ReadFile(path1)
assert.NoError(t, err)
assert.Equal(t, "1", string(v1))
v2, err := ioutil.ReadFile(path2)
require.NoError(t, err)
require.Equal(t, "2", string(v2))
v2, err := os.ReadFile(path2)
assert.NoError(t, err)
assert.Equal(t, "2", string(v2))
}
func TestProcessMainFilesInPlaceErrRead(t *testing.T) {
@@ -134,18 +132,18 @@ func TestProcessMainFilesInPlaceErrRead(t *testing.T) {
exit := p.main([]string{"/this/path/is/invalid"}, os.Stdin, os.Stdout, os.Stderr)
require.Equal(t, -1, exit)
assert.Equal(t, -1, exit)
}
func TestProcessMainFilesInPlaceFailFn(t *testing.T) {
dir, err := ioutil.TempDir("", "")
require.NoError(t, err)
dir, err := os.MkdirTemp("", "")
assert.NoError(t, err)
defer os.RemoveAll(dir)
path1 := path.Join(dir, "file1")
err = ioutil.WriteFile(path1, []byte("content 1"), 0600)
require.NoError(t, err)
err = os.WriteFile(path1, []byte("content 1"), 0600)
assert.NoError(t, err)
p := Program{
Fn: func(io.Reader, io.Writer) error { return fmt.Errorf("oh no") },
@@ -154,15 +152,15 @@ func TestProcessMainFilesInPlaceFailFn(t *testing.T) {
exit := p.main([]string{path1}, os.Stdin, os.Stdout, os.Stderr)
require.Equal(t, -1, exit)
assert.Equal(t, -1, exit)
v1, err := ioutil.ReadFile(path1)
require.NoError(t, err)
require.Equal(t, "content 1", string(v1))
v1, err := os.ReadFile(path1)
assert.NoError(t, err)
assert.Equal(t, "content 1", string(v1))
}
func dummyFileFn(r io.Reader, w io.Writer) error {
b, err := ioutil.ReadAll(r)
b, err := io.ReadAll(r)
if err != nil {
return err
}
-65
View File
@@ -1,65 +0,0 @@
package danger
import (
"fmt"
"reflect"
"unsafe"
)
const maxInt = uintptr(int(^uint(0) >> 1))
func SubsliceOffset(data []byte, subslice []byte) int {
datap := (*reflect.SliceHeader)(unsafe.Pointer(&data))
hlp := (*reflect.SliceHeader)(unsafe.Pointer(&subslice))
if hlp.Data < datap.Data {
panic(fmt.Errorf("subslice address (%d) is before data address (%d)", hlp.Data, datap.Data))
}
offset := hlp.Data - datap.Data
if offset > maxInt {
panic(fmt.Errorf("slice offset larger than int (%d)", offset))
}
intoffset := int(offset)
if intoffset > datap.Len {
panic(fmt.Errorf("slice offset (%d) is farther than data length (%d)", intoffset, datap.Len))
}
if intoffset+hlp.Len > datap.Len {
panic(fmt.Errorf("slice ends (%d+%d) is farther than data length (%d)", intoffset, hlp.Len, datap.Len))
}
return intoffset
}
func BytesRange(start []byte, end []byte) []byte {
if start == nil || end == nil {
panic("cannot call BytesRange with nil")
}
startp := (*reflect.SliceHeader)(unsafe.Pointer(&start))
endp := (*reflect.SliceHeader)(unsafe.Pointer(&end))
if startp.Data > endp.Data {
panic(fmt.Errorf("start pointer address (%d) is after end pointer address (%d)", startp.Data, endp.Data))
}
l := startp.Len
endLen := int(endp.Data-startp.Data) + endp.Len
if endLen > l {
l = endLen
}
if l > startp.Cap {
panic(fmt.Errorf("range length is larger than capacity"))
}
return start[:l]
}
func Stride(ptr unsafe.Pointer, size uintptr, offset int) unsafe.Pointer {
// TODO: replace with unsafe.Add when Go 1.17 is released
// https://github.com/golang/go/issues/40481
return unsafe.Pointer(uintptr(ptr) + uintptr(int(size)*offset))
}
-178
View File
@@ -1,178 +0,0 @@
package danger_test
import (
"testing"
"unsafe"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/danger"
)
func TestSubsliceOffsetValid(t *testing.T) {
examples := []struct {
desc string
test func() ([]byte, []byte)
offset int
}{
{
desc: "simple",
test: func() ([]byte, []byte) {
data := []byte("hello")
return data, data[1:]
},
offset: 1,
},
}
for _, e := range examples {
t.Run(e.desc, func(t *testing.T) {
d, s := e.test()
offset := danger.SubsliceOffset(d, s)
assert.Equal(t, e.offset, offset)
})
}
}
func TestSubsliceOffsetInvalid(t *testing.T) {
examples := []struct {
desc string
test func() ([]byte, []byte)
}{
{
desc: "unrelated arrays",
test: func() ([]byte, []byte) {
return []byte("one"), []byte("two")
},
},
{
desc: "slice starts before data",
test: func() ([]byte, []byte) {
full := []byte("hello world")
return full[5:], full[1:]
},
},
{
desc: "slice starts after data",
test: func() ([]byte, []byte) {
full := []byte("hello world")
return full[:3], full[5:]
},
},
{
desc: "slice ends after data",
test: func() ([]byte, []byte) {
full := []byte("hello world")
return full[:5], full[3:8]
},
},
}
for _, e := range examples {
t.Run(e.desc, func(t *testing.T) {
d, s := e.test()
require.Panics(t, func() {
danger.SubsliceOffset(d, s)
})
})
}
}
func TestStride(t *testing.T) {
a := []byte{1, 2, 3, 4}
x := &a[1]
n := (*byte)(danger.Stride(unsafe.Pointer(x), unsafe.Sizeof(byte(0)), 1))
require.Equal(t, &a[2], n)
n = (*byte)(danger.Stride(unsafe.Pointer(x), unsafe.Sizeof(byte(0)), -1))
require.Equal(t, &a[0], n)
}
func TestBytesRange(t *testing.T) {
type fn = func() ([]byte, []byte)
examples := []struct {
desc string
test fn
expected []byte
}{
{
desc: "simple",
test: func() ([]byte, []byte) {
full := []byte("hello world")
return full[1:3], full[6:8]
},
expected: []byte("ello wo"),
},
{
desc: "full",
test: func() ([]byte, []byte) {
full := []byte("hello world")
return full[0:1], full[len(full)-1:]
},
expected: []byte("hello world"),
},
{
desc: "end before start",
test: func() ([]byte, []byte) {
full := []byte("hello world")
return full[len(full)-1:], full[0:1]
},
},
{
desc: "nils",
test: func() ([]byte, []byte) {
return nil, nil
},
},
{
desc: "nils start",
test: func() ([]byte, []byte) {
return nil, []byte("foo")
},
},
{
desc: "nils end",
test: func() ([]byte, []byte) {
return []byte("foo"), nil
},
},
{
desc: "start is end",
test: func() ([]byte, []byte) {
full := []byte("hello world")
return full[1:3], full[1:3]
},
expected: []byte("el"),
},
{
desc: "end contained in start",
test: func() ([]byte, []byte) {
full := []byte("hello world")
return full[1:7], full[2:4]
},
expected: []byte("ello w"),
},
{
desc: "different backing arrays",
test: func() ([]byte, []byte) {
one := []byte("hello world")
two := []byte("hello world")
return one, two
},
},
}
for _, e := range examples {
t.Run(e.desc, func(t *testing.T) {
start, end := e.test()
if e.expected == nil {
require.Panics(t, func() {
danger.BytesRange(start, end)
})
} else {
res := danger.BytesRange(start, end)
require.Equal(t, e.expected, res)
}
})
}
}
-23
View File
@@ -1,23 +0,0 @@
package danger
import (
"reflect"
"unsafe"
)
// typeID is used as key in encoder and decoder caches to enable using
// the optimize runtime.mapaccess2_fast64 function instead of the more
// expensive lookup if we were to use reflect.Type as map key.
//
// typeID holds the pointer to the reflect.Type value, which is unique
// in the program.
//
// https://github.com/segmentio/encoding/blob/master/json/codec.go#L59-L61
type TypeID unsafe.Pointer
func MakeTypeID(t reflect.Type) TypeID {
// reflect.Type has the fields:
// typ unsafe.Pointer
// ptr unsafe.Pointer
return TypeID((*[2]unsafe.Pointer)(unsafe.Pointer(&t))[1])
}
@@ -9,7 +9,7 @@ import (
"time"
"github.com/pelletier/go-toml/v2"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
func TestDocMarshal(t *testing.T) {
@@ -107,13 +107,13 @@ name = 'List.Second'
`
result, err := toml.Marshal(docData)
require.NoError(t, err)
require.Equal(t, marshalTestToml, string(result))
assert.NoError(t, err)
assert.Equal(t, marshalTestToml, string(result))
}
func TestBasicMarshalQuotedKey(t *testing.T) {
result, err := toml.Marshal(quotedKeyMarshalTestData)
require.NoError(t, err)
assert.NoError(t, err)
expected := `'Z.string-àéù' = 'Hello'
'Yfloat-𝟘' = 3.5
@@ -128,7 +128,7 @@ String2 = 'Two'
String2 = 'Three'
`
require.Equal(t, string(expected), string(result))
assert.Equal(t, string(expected), string(result))
}
@@ -153,7 +153,7 @@ func TestEmptyMarshal(t *testing.T) {
Map: map[string]string{},
}
result, err := toml.Marshal(doc)
require.NoError(t, err)
assert.NoError(t, err)
expected := `title = 'Placeholder'
bool = false
@@ -164,7 +164,7 @@ stringlist = []
[map]
`
require.Equal(t, string(expected), string(result))
assert.Equal(t, string(expected), string(result))
}
type textMarshaler struct {
@@ -187,13 +187,13 @@ func TestTextMarshaler(t *testing.T) {
t.Run("at root", func(t *testing.T) {
_, err := toml.Marshal(m)
// in v2 we do not allow TextMarshaler at root
require.Error(t, err)
assert.Error(t, err)
})
t.Run("leaf", func(t *testing.T) {
res, err := toml.Marshal(wrap{m})
require.NoError(t, err)
assert.NoError(t, err)
require.Equal(t, "TM = 'Sally Fields'\n", string(res))
assert.Equal(t, "TM = 'Sally Fields'\n", string(res))
})
}
@@ -16,8 +16,7 @@ import (
"time"
"github.com/pelletier/go-toml/v2"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
type basicMarshalTestStruct struct {
@@ -123,7 +122,7 @@ func TestInterface(t *testing.T) {
var config Conf
config.Inter = &NestedStruct{}
err := toml.Unmarshal(doc, &config)
require.NoError(t, err)
assert.NoError(t, err)
expected := Conf{
Name: "rui",
Age: 18,
@@ -139,8 +138,8 @@ func TestInterface(t *testing.T) {
func TestBasicUnmarshal(t *testing.T) {
result := basicMarshalTestStruct{}
err := toml.Unmarshal(basicTestToml, &result)
require.NoError(t, err)
require.Equal(t, basicTestData, result)
assert.NoError(t, err)
assert.Equal(t, basicTestData, result)
}
type quotedKeyMarshalTestStruct struct {
@@ -300,7 +299,7 @@ func TestDocUnmarshal(t *testing.T) {
result := testDoc{}
err := toml.Unmarshal(marshalTestToml, &result)
expected := docData
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, expected, result)
}
@@ -340,7 +339,7 @@ shouldntBeHere = 2
func TestUnexportedUnmarshal(t *testing.T) {
result := unexportedMarshalTestStruct{}
err := toml.Unmarshal(unexportedTestToml, &result)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, unexportedTestData, result)
}
@@ -456,7 +455,7 @@ func TestEmptytomlUnmarshal(t *testing.T) {
result := emptyMarshalTestStruct{}
err := toml.Unmarshal(emptyTestToml, &result)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, emptyTestData, result)
}
@@ -504,7 +503,7 @@ Str = "Hello"
func TestPointerUnmarshal(t *testing.T) {
result := pointerMarshalTestStruct{}
err := toml.Unmarshal(pointerTestToml, &result)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, pointerTestData, result)
}
@@ -540,7 +539,7 @@ StringPtr = [["Three", "Four"]]
func TestNestedUnmarshal(t *testing.T) {
result := nestedMarshalTestStruct{}
err := toml.Unmarshal(nestedTestToml, &result)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, nestedTestData, result)
}
@@ -834,7 +833,7 @@ func TestUnmarshalTabInStringAndQuotedKey(t *testing.T) {
t.Run(test.desc, func(t *testing.T) {
result := Test{}
err := toml.Unmarshal(test.input, &result)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, test.expected, result)
})
}
@@ -963,7 +962,7 @@ func TestUnmarshalTypeTableHeader(t *testing.T) {
}
expected := map[header]map[string]int{
"test": map[string]int{"a": 1},
"test": {"a": 1},
}
if !reflect.DeepEqual(result, expected) {
@@ -1090,7 +1089,7 @@ func TestUnmarshalCheckConversionFloatInt(t *testing.T) {
for _, test := range testCases {
t.Run(test.desc, func(t *testing.T) {
err := toml.Unmarshal([]byte(test.input), &conversionCheck{})
require.Error(t, err)
assert.Error(t, err)
})
}
}
@@ -1125,7 +1124,7 @@ func TestUnmarshalOverflow(t *testing.T) {
for _, test := range testCases {
t.Run(test.desc, func(t *testing.T) {
err := toml.Unmarshal([]byte(test.input), &overflow{})
require.Error(t, err)
assert.Error(t, err)
})
}
}
@@ -1745,7 +1744,7 @@ Age = 23
}
actual := OuterStruct{}
err := toml.Unmarshal(doc, &actual)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, expected, actual)
}
@@ -1830,7 +1829,7 @@ InnerField = "After4"
}
err := toml.Unmarshal(doc, &actual)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, expected, actual)
}
@@ -1879,7 +1878,7 @@ type arrayTooSmallStruct struct {
func TestUnmarshalSlice(t *testing.T) {
var actual sliceStruct
err := toml.Unmarshal(sliceTomlDemo, &actual)
require.NoError(t, err)
assert.NoError(t, err)
expected := sliceStruct{
Slice: []string{"Howdy", "Hey There"},
SlicePtr: &[]string{"Howdy", "Hey There"},
@@ -1930,7 +1929,7 @@ func TestUnmarshalMixedTypeSlice(t *testing.T) {
},
}
err := toml.Unmarshal(doc, &actual)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, expected, actual)
}
@@ -1939,7 +1938,7 @@ func TestUnmarshalArray(t *testing.T) {
var actual arrayStruct
err = toml.Unmarshal(sliceTomlDemo, &actual)
require.NoError(t, err)
assert.NoError(t, err)
expected := arrayStruct{
Slice: [4]string{"Howdy", "Hey There"},
@@ -1998,8 +1997,13 @@ func TestDecoderStrict(t *testing.T) {
}
err := strictDecoder(input).Decode(&doc)
require.Error(t, err)
require.IsType(t, &toml.StrictMissingError{}, err)
assert.Error(t, err)
assert.Equal(t,
reflect.TypeOf(err), reflect.TypeOf(&toml.StrictMissingError{}),
"Expected a *toml.StrictMissingError, got: %v", reflect.TypeOf(err),
)
se := err.(*toml.StrictMissingError)
keys := []toml.Key{}
@@ -2015,10 +2019,10 @@ func TestDecoderStrict(t *testing.T) {
{"undecoded", "array"},
}
require.Equal(t, expectedKeys, keys)
assert.Equal(t, expectedKeys, keys)
err = decoder(input).Decode(&doc)
require.NoError(t, err)
assert.NoError(t, err)
var m map[string]interface{}
err = decoder(input).Decode(&m)
@@ -2036,7 +2040,7 @@ func TestDecoderStrictValid(t *testing.T) {
}
err := strictDecoder(input).Decode(&doc)
require.NoError(t, err)
assert.NoError(t, err)
}
type docUnmarshalTOML struct {
@@ -2087,7 +2091,7 @@ func TestCustomUnmarshal(t *testing.T) {
var d parent
err := toml.Unmarshal([]byte(input), &d)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, "ok1", d.Doc.Decoded.Key)
assert.Equal(t, "ok2", d.DocPointer.Decoded.Key)
}
@@ -2153,7 +2157,7 @@ Int = 21
Float = 2.0
`
err := toml.Unmarshal([]byte(input), &doc)
require.NoError(t, err)
assert.NoError(t, err)
assert.Equal(t, 12, doc.UnixTime.Value)
assert.Equal(t, 42, doc.Version.Value)
assert.Equal(t, 1, doc.Bool.Value)
@@ -2223,7 +2227,10 @@ func TestUnmarshalEmptyInterface(t *testing.T) {
if err != nil {
t.Fatal(err)
}
require.IsType(t, map[string]interface{}{}, v)
assert.Equal(t,
reflect.TypeOf(map[string]interface{}{}), reflect.TypeOf(v),
"Expected map[string]interface{}{} type, got: %v", reflect.TypeOf(v),
)
x := v.(map[string]interface{})
assert.Equal(t, "pelletier", x["User"])
@@ -2276,7 +2283,7 @@ func (c *Custom) UnmarshalTOML(v interface{}) error {
return nil
}
func TestGithubIssue431(t *testing.T) {
func TestGitHubIssue431(t *testing.T) {
doc := `key = "value"`
var c Config
if err := toml.Unmarshal([]byte(doc), &c); err != nil {
@@ -2314,7 +2321,7 @@ type config437 struct {
} `toml:"HTTP"`
}
func TestGithubIssue437(t *testing.T) {
func TestGitHubIssue437(t *testing.T) {
t.Skipf("unmarshalTOML not implemented")
src := `
[HTTP]
+24 -5
View File
@@ -4,6 +4,8 @@ import (
"fmt"
"strconv"
"time"
"github.com/pelletier/go-toml/v2"
)
// Remove JSON tags to a data structure as returned by toml-test.
@@ -40,7 +42,7 @@ func rmTag(typedJson interface{}) (interface{}, error) {
}
return m, nil
// Array: remove tags from all itenm.
// Array: remove tags from all items.
case []interface{}:
a := make([]interface{}, len(v))
for i := range v {
@@ -76,14 +78,31 @@ func untag(typed map[string]interface{}) (interface{}, error) {
return nil, fmt.Errorf("untag: %w", err)
}
return f, nil
//toml.LocalDate{Year:2020, Month:12, Day:12}
case "datetime":
return parseTime(v, "2006-01-02T15:04:05.999999999Z07:00", false)
return time.Parse("2006-01-02T15:04:05.999999999Z07:00", v)
case "datetime-local":
return parseTime(v, "2006-01-02T15:04:05.999999999", true)
var t toml.LocalDateTime
err := t.UnmarshalText([]byte(v))
if err != nil {
return nil, fmt.Errorf("untag: %w", err)
}
return t, nil
case "date-local":
return parseTime(v, "2006-01-02", true)
var t toml.LocalDate
err := t.UnmarshalText([]byte(v))
if err != nil {
return nil, fmt.Errorf("untag: %w", err)
}
return t, nil
case "time-local":
return parseTime(v, "15:04:05.999999999", true)
var t toml.LocalTime
err := t.UnmarshalText([]byte(v))
if err != nil {
return nil, fmt.Errorf("untag: %w", err)
}
return t, nil
case "bool":
switch v {
case "true":
+19 -1
View File
@@ -10,7 +10,7 @@ import (
"github.com/pelletier/go-toml/v2"
)
// Marshal is a helpfer function for calling toml.Marshal
// Marshal is a helper function for calling toml.Marshal
//
// Only needed to avoid package import loops.
func Marshal(v interface{}) ([]byte, error) {
@@ -48,3 +48,21 @@ func DecodeStdin() error {
return nil
}
// EncodeStdin is a helper function for the toml-test binary interface. Tagged
// JSON is read from STDIN and a resulting TOML representation is written to
// STDOUT.
func EncodeStdin() error {
var j interface{}
err := json.NewDecoder(os.Stdin).Decode(&j)
if err != nil {
return err
}
rm, err := rmTag(j)
if err != nil {
return fmt.Errorf("removing tags: %w", err)
}
return toml.NewEncoder(os.Stdout).Encode(rm)
}
+38 -36
View File
@@ -57,7 +57,11 @@ type SeenTracker struct {
currentIdx int
}
var pool sync.Pool
var pool = sync.Pool{
New: func() interface{} {
return &SeenTracker{}
},
}
func (s *SeenTracker) reset() {
// Always contains a root element at index 0.
@@ -149,8 +153,9 @@ func (s *SeenTracker) setExplicitFlag(parentIdx int) {
// CheckExpression takes a top-level node and checks that it does not contain
// keys that have been seen in previous calls, and validates that types are
// consistent.
func (s *SeenTracker) CheckExpression(node *unstable.Node) error {
// consistent. It returns true if it is the first time this node's key is seen.
// Useful to clear array tables on first use.
func (s *SeenTracker) CheckExpression(node *unstable.Node) (bool, error) {
if s.entries == nil {
s.reset()
}
@@ -166,7 +171,7 @@ func (s *SeenTracker) CheckExpression(node *unstable.Node) error {
}
}
func (s *SeenTracker) checkTable(node *unstable.Node) error {
func (s *SeenTracker) checkTable(node *unstable.Node) (bool, error) {
if s.currentIdx >= 0 {
s.setExplicitFlag(s.currentIdx)
}
@@ -192,7 +197,7 @@ func (s *SeenTracker) checkTable(node *unstable.Node) error {
} else {
entry := s.entries[idx]
if entry.kind == valueKind {
return fmt.Errorf("toml: expected %s to be a table, not a %s", string(k), entry.kind)
return false, fmt.Errorf("toml: expected %s to be a table, not a %s", string(k), entry.kind)
}
}
parentIdx = idx
@@ -201,25 +206,27 @@ func (s *SeenTracker) checkTable(node *unstable.Node) error {
k := it.Node().Data
idx := s.find(parentIdx, k)
first := false
if idx >= 0 {
kind := s.entries[idx].kind
if kind != tableKind {
return fmt.Errorf("toml: key %s should be a table, not a %s", string(k), kind)
return false, fmt.Errorf("toml: key %s should be a table, not a %s", string(k), kind)
}
if s.entries[idx].explicit {
return fmt.Errorf("toml: table %s already exists", string(k))
return false, fmt.Errorf("toml: table %s already exists", string(k))
}
s.entries[idx].explicit = true
} else {
idx = s.create(parentIdx, k, tableKind, true, false)
first = true
}
s.currentIdx = idx
return nil
return first, nil
}
func (s *SeenTracker) checkArrayTable(node *unstable.Node) error {
func (s *SeenTracker) checkArrayTable(node *unstable.Node) (bool, error) {
if s.currentIdx >= 0 {
s.setExplicitFlag(s.currentIdx)
}
@@ -242,7 +249,7 @@ func (s *SeenTracker) checkArrayTable(node *unstable.Node) error {
} else {
entry := s.entries[idx]
if entry.kind == valueKind {
return fmt.Errorf("toml: expected %s to be a table, not a %s", string(k), entry.kind)
return false, fmt.Errorf("toml: expected %s to be a table, not a %s", string(k), entry.kind)
}
}
@@ -252,22 +259,23 @@ func (s *SeenTracker) checkArrayTable(node *unstable.Node) error {
k := it.Node().Data
idx := s.find(parentIdx, k)
if idx >= 0 {
firstTime := idx < 0
if firstTime {
idx = s.create(parentIdx, k, arrayTableKind, true, false)
} else {
kind := s.entries[idx].kind
if kind != arrayTableKind {
return fmt.Errorf("toml: key %s already exists as a %s, but should be an array table", kind, string(k))
return false, fmt.Errorf("toml: key %s already exists as a %s, but should be an array table", kind, string(k))
}
s.clear(idx)
} else {
idx = s.create(parentIdx, k, arrayTableKind, true, false)
}
s.currentIdx = idx
return nil
return firstTime, nil
}
func (s *SeenTracker) checkKeyValue(node *unstable.Node) error {
func (s *SeenTracker) checkKeyValue(node *unstable.Node) (bool, error) {
parentIdx := s.currentIdx
it := node.Key()
@@ -281,11 +289,11 @@ func (s *SeenTracker) checkKeyValue(node *unstable.Node) error {
} else {
entry := s.entries[idx]
if it.IsLast() {
return fmt.Errorf("toml: key %s is already defined", string(k))
return false, fmt.Errorf("toml: key %s is already defined", string(k))
} else if entry.kind != tableKind {
return fmt.Errorf("toml: expected %s to be a table, not a %s", string(k), entry.kind)
return false, fmt.Errorf("toml: expected %s to be a table, not a %s", string(k), entry.kind)
} else if entry.explicit {
return fmt.Errorf("toml: cannot redefine table %s that has already been explicitly defined", string(k))
return false, fmt.Errorf("toml: cannot redefine table %s that has already been explicitly defined", string(k))
}
}
@@ -303,45 +311,39 @@ func (s *SeenTracker) checkKeyValue(node *unstable.Node) error {
return s.checkArray(value)
}
return nil
return false, nil
}
func (s *SeenTracker) checkArray(node *unstable.Node) error {
func (s *SeenTracker) checkArray(node *unstable.Node) (first bool, err error) {
it := node.Children()
for it.Next() {
n := it.Node()
switch n.Kind {
case unstable.InlineTable:
err := s.checkInlineTable(n)
first, err = s.checkInlineTable(n)
if err != nil {
return err
return false, err
}
case unstable.Array:
err := s.checkArray(n)
first, err = s.checkArray(n)
if err != nil {
return err
return false, err
}
}
}
return nil
return first, nil
}
func (s *SeenTracker) checkInlineTable(node *unstable.Node) error {
if pool.New == nil {
pool.New = func() interface{} {
return &SeenTracker{}
}
}
func (s *SeenTracker) checkInlineTable(node *unstable.Node) (first bool, err error) {
s = pool.Get().(*SeenTracker)
s.reset()
it := node.Children()
for it.Next() {
n := it.Node()
err := s.checkKeyValue(n)
first, err = s.checkKeyValue(n)
if err != nil {
return err
return false, err
}
}
@@ -352,5 +354,5 @@ func (s *SeenTracker) checkInlineTable(node *unstable.Node) error {
// redefinition of its keys: check* functions cannot walk into
// a value.
pool.Put(s)
return nil
return first, nil
}
+7 -3
View File
@@ -1,10 +1,10 @@
package tracker
import (
"reflect"
"testing"
"unsafe"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
func TestEntrySize(t *testing.T) {
@@ -12,5 +12,9 @@ func TestEntrySize(t *testing.T) {
// performance of unmarshaling documents. Should only be increased with care
// and a very good reason.
maxExpectedEntrySize := 48
require.LessOrEqual(t, int(unsafe.Sizeof(entry{})), maxExpectedEntrySize)
assert.True(t,
int(reflect.TypeOf(entry{}).Size()) <= maxExpectedEntrySize,
"Expected entry to be less than or equal to %d, got: %d",
maxExpectedEntrySize, int(reflect.TypeOf(entry{}).Size()),
)
}
+28 -28
View File
@@ -5,73 +5,73 @@ import (
"time"
"github.com/pelletier/go-toml/v2"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
func TestLocalDate_AsTime(t *testing.T) {
d := toml.LocalDate{2021, 6, 8}
cast := d.AsTime(time.UTC)
require.Equal(t, time.Date(2021, time.June, 8, 0, 0, 0, 0, time.UTC), cast)
assert.Equal(t, time.Date(2021, time.June, 8, 0, 0, 0, 0, time.UTC), cast)
}
func TestLocalDate_String(t *testing.T) {
d := toml.LocalDate{2021, 6, 8}
require.Equal(t, "2021-06-08", d.String())
assert.Equal(t, "2021-06-08", d.String())
}
func TestLocalDate_MarshalText(t *testing.T) {
d := toml.LocalDate{2021, 6, 8}
b, err := d.MarshalText()
require.NoError(t, err)
require.Equal(t, []byte("2021-06-08"), b)
assert.NoError(t, err)
assert.Equal(t, []byte("2021-06-08"), b)
}
func TestLocalDate_UnmarshalMarshalText(t *testing.T) {
d := toml.LocalDate{}
err := d.UnmarshalText([]byte("2021-06-08"))
require.NoError(t, err)
require.Equal(t, toml.LocalDate{2021, 6, 8}, d)
assert.NoError(t, err)
assert.Equal(t, toml.LocalDate{2021, 6, 8}, d)
err = d.UnmarshalText([]byte("what"))
require.Error(t, err)
assert.Error(t, err)
}
func TestLocalTime_String(t *testing.T) {
d := toml.LocalTime{20, 12, 1, 2, 9}
require.Equal(t, "20:12:01.000000002", d.String())
assert.Equal(t, "20:12:01.000000002", d.String())
d = toml.LocalTime{20, 12, 1, 0, 0}
require.Equal(t, "20:12:01", d.String())
assert.Equal(t, "20:12:01", d.String())
d = toml.LocalTime{20, 12, 1, 0, 9}
require.Equal(t, "20:12:01.000000000", d.String())
assert.Equal(t, "20:12:01.000000000", d.String())
d = toml.LocalTime{20, 12, 1, 100, 0}
require.Equal(t, "20:12:01.0000001", d.String())
assert.Equal(t, "20:12:01.0000001", d.String())
}
func TestLocalTime_MarshalText(t *testing.T) {
d := toml.LocalTime{20, 12, 1, 2, 9}
b, err := d.MarshalText()
require.NoError(t, err)
require.Equal(t, []byte("20:12:01.000000002"), b)
assert.NoError(t, err)
assert.Equal(t, []byte("20:12:01.000000002"), b)
}
func TestLocalTime_UnmarshalMarshalText(t *testing.T) {
d := toml.LocalTime{}
err := d.UnmarshalText([]byte("20:12:01.000000002"))
require.NoError(t, err)
require.Equal(t, toml.LocalTime{20, 12, 1, 2, 9}, d)
assert.NoError(t, err)
assert.Equal(t, toml.LocalTime{20, 12, 1, 2, 9}, d)
err = d.UnmarshalText([]byte("what"))
require.Error(t, err)
assert.Error(t, err)
err = d.UnmarshalText([]byte("20:12:01.000000002 bad"))
require.Error(t, err)
assert.Error(t, err)
}
func TestLocalTime_RoundTrip(t *testing.T) {
var d struct{ A toml.LocalTime }
err := toml.Unmarshal([]byte("a=20:12:01.500"), &d)
require.NoError(t, err)
require.Equal(t, "20:12:01.500", d.A.String())
assert.NoError(t, err)
assert.Equal(t, "20:12:01.500", d.A.String())
}
func TestLocalDateTime_AsTime(t *testing.T) {
@@ -80,7 +80,7 @@ func TestLocalDateTime_AsTime(t *testing.T) {
toml.LocalTime{20, 12, 1, 2, 9},
}
cast := d.AsTime(time.UTC)
require.Equal(t, time.Date(2021, time.June, 8, 20, 12, 1, 2, time.UTC), cast)
assert.Equal(t, time.Date(2021, time.June, 8, 20, 12, 1, 2, time.UTC), cast)
}
func TestLocalDateTime_String(t *testing.T) {
@@ -88,7 +88,7 @@ func TestLocalDateTime_String(t *testing.T) {
toml.LocalDate{2021, 6, 8},
toml.LocalTime{20, 12, 1, 2, 9},
}
require.Equal(t, "2021-06-08T20:12:01.000000002", d.String())
assert.Equal(t, "2021-06-08T20:12:01.000000002", d.String())
}
func TestLocalDateTime_MarshalText(t *testing.T) {
@@ -97,22 +97,22 @@ func TestLocalDateTime_MarshalText(t *testing.T) {
toml.LocalTime{20, 12, 1, 2, 9},
}
b, err := d.MarshalText()
require.NoError(t, err)
require.Equal(t, []byte("2021-06-08T20:12:01.000000002"), b)
assert.NoError(t, err)
assert.Equal(t, []byte("2021-06-08T20:12:01.000000002"), b)
}
func TestLocalDateTime_UnmarshalMarshalText(t *testing.T) {
d := toml.LocalDateTime{}
err := d.UnmarshalText([]byte("2021-06-08 20:12:01.000000002"))
require.NoError(t, err)
require.Equal(t, toml.LocalDateTime{
assert.NoError(t, err)
assert.Equal(t, toml.LocalDateTime{
toml.LocalDate{2021, 6, 8},
toml.LocalTime{20, 12, 1, 2, 9},
}, d)
err = d.UnmarshalText([]byte("what"))
require.Error(t, err)
assert.Error(t, err)
err = d.UnmarshalText([]byte("2021-06-08 20:12:01.000000002 bad"))
require.Error(t, err)
assert.Error(t, err)
}
+146 -21
View File
@@ -3,11 +3,12 @@ package toml
import (
"bytes"
"encoding"
"encoding/json"
"fmt"
"io"
"math"
"reflect"
"sort"
"slices"
"strconv"
"strings"
"time"
@@ -37,10 +38,11 @@ type Encoder struct {
w io.Writer
// global settings
tablesInline bool
arraysMultiline bool
indentSymbol string
indentTables bool
tablesInline bool
arraysMultiline bool
indentSymbol string
indentTables bool
marshalJsonNumbers bool
}
// NewEncoder returns a new Encoder that writes to w.
@@ -87,6 +89,17 @@ func (enc *Encoder) SetIndentTables(indent bool) *Encoder {
return enc
}
// SetMarshalJsonNumbers forces the encoder to serialize `json.Number` as a
// float or integer instead of relying on TextMarshaler to emit a string.
//
// *Unstable:* This method does not follow the compatibility guarantees of
// semver. It can be changed or removed without a new major version being
// issued.
func (enc *Encoder) SetMarshalJsonNumbers(indent bool) *Encoder {
enc.marshalJsonNumbers = indent
return enc
}
// Encode writes a TOML representation of v to the stream.
//
// If v cannot be represented to TOML it returns an error.
@@ -148,6 +161,11 @@ func (enc *Encoder) SetIndentTables(indent bool) *Encoder {
//
// The "omitempty" option prevents empty values or groups from being emitted.
//
// The "omitzero" option prevents zero values or groups from being emitted.
//
// The "commented" option prefixes the value and all its children with a comment
// symbol.
//
// In addition to the "toml" tag struct tag, a "comment" tag can be used to emit
// a TOML comment before the value being annotated. Comments are ignored inside
// inline tables. For array tables, the comment is only present before the first
@@ -180,6 +198,8 @@ func (enc *Encoder) Encode(v interface{}) error {
type valueOptions struct {
multiline bool
omitempty bool
omitzero bool
commented bool
comment string
}
@@ -205,6 +225,9 @@ type encoderCtx struct {
// Indentation level
indent int
// Prefix the current value with a comment.
commented bool
// Options coming from struct tags
options valueOptions
}
@@ -245,10 +268,22 @@ func (enc *Encoder) encode(b []byte, ctx encoderCtx, v reflect.Value) ([]byte, e
return append(b, x.String()...), nil
case LocalDateTime:
return append(b, x.String()...), nil
case json.Number:
if enc.marshalJsonNumbers {
if x == "" { /// Useful zero value.
return append(b, "0"...), nil
} else if v, err := x.Int64(); err == nil {
return enc.encode(b, ctx, reflect.ValueOf(v))
} else if f, err := x.Float64(); err == nil {
return enc.encode(b, ctx, reflect.ValueOf(f))
} else {
return nil, fmt.Errorf("toml: unable to convert %q to int64 or float64", x)
}
}
}
hasTextMarshaler := v.Type().Implements(textMarshalerType)
if hasTextMarshaler || (v.CanAddr() && reflect.PtrTo(v.Type()).Implements(textMarshalerType)) {
if hasTextMarshaler || (v.CanAddr() && reflect.PointerTo(v.Type()).Implements(textMarshalerType)) {
if !hasTextMarshaler {
v = v.Addr()
}
@@ -273,7 +308,7 @@ func (enc *Encoder) encode(b []byte, ctx encoderCtx, v reflect.Value) ([]byte, e
return enc.encodeMap(b, ctx, v)
case reflect.Struct:
return enc.encodeStruct(b, ctx, v)
case reflect.Slice:
case reflect.Slice, reflect.Array:
return enc.encodeSlice(b, ctx, v)
case reflect.Interface:
if v.IsNil() {
@@ -352,11 +387,16 @@ func shouldOmitEmpty(options valueOptions, v reflect.Value) bool {
return options.omitempty && isEmptyValue(v)
}
func shouldOmitZero(options valueOptions, v reflect.Value) bool {
return options.omitzero && v.IsZero()
}
func (enc *Encoder) encodeKv(b []byte, ctx encoderCtx, options valueOptions, v reflect.Value) ([]byte, error) {
var err error
if !ctx.inline {
b = enc.encodeComment(ctx.indent, options.comment, b)
b = enc.commented(ctx.commented, b)
b = enc.indent(ctx.indent, b)
}
@@ -378,6 +418,13 @@ func (enc *Encoder) encodeKv(b []byte, ctx encoderCtx, options valueOptions, v r
return b, nil
}
func (enc *Encoder) commented(commented bool, b []byte) []byte {
if commented {
return append(b, "# "...)
}
return b
}
func isEmptyValue(v reflect.Value) bool {
switch v.Kind() {
case reflect.Struct:
@@ -477,12 +524,26 @@ func (enc *Encoder) encodeQuotedString(multiline bool, b []byte, v string) []byt
del = 0x7f
)
for _, r := range []byte(v) {
bv := []byte(v)
for i := 0; i < len(bv); i++ {
r := bv[i]
switch r {
case '\\':
b = append(b, `\\`...)
case '"':
b = append(b, `\"`...)
if multiline {
// Quotation marks do not need to be quoted in multiline strings unless
// it contains 3 consecutive. If 3+ quotes appear, quote all of them
// because it's visually better
if i+2 > len(bv) || bv[i+1] != '"' || bv[i+2] != '"' {
b = append(b, r)
} else {
b = append(b, `\"\"\"`...)
i += 2
}
} else {
b = append(b, `\"`...)
}
case '\b':
b = append(b, `\b`...)
case '\f':
@@ -526,6 +587,8 @@ func (enc *Encoder) encodeTableHeader(ctx encoderCtx, b []byte) ([]byte, error)
b = enc.encodeComment(ctx.indent, ctx.options.comment, b)
b = enc.commented(ctx.commented, b)
b = enc.indent(ctx.indent, b)
b = append(b, '[')
@@ -577,11 +640,35 @@ func (enc *Encoder) encodeKey(b []byte, k string) []byte {
}
}
func (enc *Encoder) encodeMap(b []byte, ctx encoderCtx, v reflect.Value) ([]byte, error) {
if v.Type().Key().Kind() != reflect.String {
return nil, fmt.Errorf("toml: type %s is not supported as a map key", v.Type().Key().Kind())
}
func (enc *Encoder) keyToString(k reflect.Value) (string, error) {
keyType := k.Type()
switch {
case keyType.Kind() == reflect.String:
return k.String(), nil
case keyType.Implements(textMarshalerType):
keyB, err := k.Interface().(encoding.TextMarshaler).MarshalText()
if err != nil {
return "", fmt.Errorf("toml: error marshalling key %v from text: %w", k, err)
}
return string(keyB), nil
case keyType.Kind() == reflect.Int || keyType.Kind() == reflect.Int8 || keyType.Kind() == reflect.Int16 || keyType.Kind() == reflect.Int32 || keyType.Kind() == reflect.Int64:
return strconv.FormatInt(k.Int(), 10), nil
case keyType.Kind() == reflect.Uint || keyType.Kind() == reflect.Uint8 || keyType.Kind() == reflect.Uint16 || keyType.Kind() == reflect.Uint32 || keyType.Kind() == reflect.Uint64:
return strconv.FormatUint(k.Uint(), 10), nil
case keyType.Kind() == reflect.Float32:
return strconv.FormatFloat(k.Float(), 'f', -1, 32), nil
case keyType.Kind() == reflect.Float64:
return strconv.FormatFloat(k.Float(), 'f', -1, 64), nil
}
return "", fmt.Errorf("toml: type %s is not supported as a map key", keyType.Kind())
}
func (enc *Encoder) encodeMap(b []byte, ctx encoderCtx, v reflect.Value) ([]byte, error) {
var (
t table
emptyValueOptions valueOptions
@@ -589,13 +676,17 @@ func (enc *Encoder) encodeMap(b []byte, ctx encoderCtx, v reflect.Value) ([]byte
iter := v.MapRange()
for iter.Next() {
k := iter.Key().String()
v := iter.Value()
if isNil(v) {
continue
}
k, err := enc.keyToString(iter.Key())
if err != nil {
return nil, err
}
if willConvertToTableOrArrayTable(ctx, v) {
t.pushTable(k, v, emptyValueOptions)
} else {
@@ -610,8 +701,8 @@ func (enc *Encoder) encodeMap(b []byte, ctx encoderCtx, v reflect.Value) ([]byte
}
func sortEntriesByKey(e []entry) {
sort.Slice(e, func(i, j int) bool {
return e[i].Key < e[j].Key
slices.SortFunc(e, func(a, b entry) int {
return strings.Compare(a.Key, b.Key)
})
}
@@ -674,6 +765,8 @@ func walkStruct(ctx encoderCtx, t *table, v reflect.Value) {
if fieldType.Anonymous {
if fieldType.Type.Kind() == reflect.Struct {
walkStruct(ctx, t, f)
} else if fieldType.Type.Kind() == reflect.Ptr && !f.IsNil() && f.Elem().Kind() == reflect.Struct {
walkStruct(ctx, t, f.Elem())
}
continue
} else {
@@ -688,6 +781,8 @@ func walkStruct(ctx encoderCtx, t *table, v reflect.Value) {
options := valueOptions{
multiline: opts.multiline,
omitempty: opts.omitempty,
omitzero: opts.omitzero,
commented: opts.commented,
comment: fieldType.Tag.Get("comment"),
}
@@ -747,6 +842,8 @@ type tagOptions struct {
multiline bool
inline bool
omitempty bool
omitzero bool
commented bool
}
func parseTag(tag string) (string, tagOptions) {
@@ -758,7 +855,7 @@ func parseTag(tag string) (string, tagOptions) {
}
raw := tag[idx+1:]
tag = string(tag[:idx])
tag = tag[:idx]
for raw != "" {
var o string
i := strings.Index(raw, ",")
@@ -774,6 +871,10 @@ func parseTag(tag string) (string, tagOptions) {
opts.inline = true
case "omitempty":
opts.omitempty = true
case "omitzero":
opts.omitzero = true
case "commented":
opts.commented = true
}
}
@@ -806,11 +907,16 @@ func (enc *Encoder) encodeTable(b []byte, ctx encoderCtx, t table) ([]byte, erro
if shouldOmitEmpty(kv.Options, kv.Value) {
continue
}
if shouldOmitZero(kv.Options, kv.Value) {
continue
}
hasNonEmptyKV = true
ctx.setKey(kv.Key)
ctx2 := ctx
ctx2.commented = kv.Options.commented || ctx2.commented
b, err = enc.encodeKv(b, ctx, kv.Options, kv.Value)
b, err = enc.encodeKv(b, ctx2, kv.Options, kv.Value)
if err != nil {
return nil, err
}
@@ -823,6 +929,9 @@ func (enc *Encoder) encodeTable(b []byte, ctx encoderCtx, t table) ([]byte, erro
if shouldOmitEmpty(table.Options, table.Value) {
continue
}
if shouldOmitZero(table.Options, table.Value) {
continue
}
if first {
first = false
if hasNonEmptyKV {
@@ -835,8 +944,10 @@ func (enc *Encoder) encodeTable(b []byte, ctx encoderCtx, t table) ([]byte, erro
ctx.setKey(table.Key)
ctx.options = table.Options
ctx2 := ctx
ctx2.commented = ctx2.commented || ctx.options.commented
b, err = enc.encode(b, ctx, table.Value)
b, err = enc.encode(b, ctx2, table.Value)
if err != nil {
return nil, err
}
@@ -855,6 +966,9 @@ func (enc *Encoder) encodeTableInline(b []byte, ctx encoderCtx, t table) ([]byte
if shouldOmitEmpty(kv.Options, kv.Value) {
continue
}
if shouldOmitZero(kv.Options, kv.Value) {
continue
}
if first {
first = false
@@ -883,7 +997,7 @@ func willConvertToTable(ctx encoderCtx, v reflect.Value) bool {
if !v.IsValid() {
return false
}
if v.Type() == timeType || v.Type().Implements(textMarshalerType) || (v.Kind() != reflect.Ptr && v.CanAddr() && reflect.PtrTo(v.Type()).Implements(textMarshalerType)) {
if v.Type() == timeType || v.Type().Implements(textMarshalerType) || (v.Kind() != reflect.Ptr && v.CanAddr() && reflect.PointerTo(v.Type()).Implements(textMarshalerType)) {
return false
}
@@ -914,7 +1028,7 @@ func willConvertToTableOrArrayTable(ctx encoderCtx, v reflect.Value) bool {
return willConvertToTableOrArrayTable(ctx, v.Elem())
}
if t.Kind() == reflect.Slice {
if t.Kind() == reflect.Slice || t.Kind() == reflect.Array {
if v.Len() == 0 {
// An empty slice should be a kv = [].
return false
@@ -954,6 +1068,13 @@ func (enc *Encoder) encodeSliceAsArrayTable(b []byte, ctx encoderCtx, v reflect.
ctx.shiftKey()
scratch := make([]byte, 0, 64)
scratch = enc.commented(ctx.commented, scratch)
if enc.indentTables {
scratch = enc.indent(ctx.indent, scratch)
}
scratch = append(scratch, "[["...)
for i, k := range ctx.parentKey {
@@ -969,6 +1090,10 @@ func (enc *Encoder) encodeSliceAsArrayTable(b []byte, ctx encoderCtx, v reflect.
b = enc.encodeComment(ctx.indent, ctx.options.comment, b)
if enc.indentTables {
ctx.indent++
}
for i := 0; i < v.Len(); i++ {
if i != 0 {
b = append(b, "\n"...)
+737 -60
View File
File diff suppressed because it is too large Load Diff
+42
View File
@@ -0,0 +1,42 @@
package ossfuzz
import (
"fmt"
"reflect"
"strings"
"github.com/pelletier/go-toml/v2"
)
func FuzzToml(data []byte) int {
if len(data) >= 2048 {
return 0
}
if strings.Contains(string(data), "nan") {
return 0
}
var v interface{}
err := toml.Unmarshal(data, &v)
if err != nil {
return 0
}
encoded, err := toml.Marshal(v)
if err != nil {
panic(fmt.Sprintf("failed to marshal unmarshaled document: %s", err))
}
var v2 interface{}
err = toml.Unmarshal(encoded, &v2)
if err != nil {
panic(fmt.Sprintf("failed round trip: %s", err))
}
if !reflect.DeepEqual(v, v2) {
panic(fmt.Sprintf("not equal: %#+v %#+v", v, v2))
}
return 1
}
+1 -2
View File
@@ -1,7 +1,6 @@
package toml
import (
"github.com/pelletier/go-toml/v2/internal/danger"
"github.com/pelletier/go-toml/v2/internal/tracker"
"github.com/pelletier/go-toml/v2/unstable"
)
@@ -103,5 +102,5 @@ func keyLocation(node *unstable.Node) []byte {
end = k.Node().Data
}
return danger.BytesRange(start, end)
return start[:cap(start)-cap(end)+len(end)]
}
+596
View File
@@ -0,0 +1,596 @@
#!/usr/bin/env bash
set -uo pipefail
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Go versions to test (1.11 through 1.25)
GO_VERSIONS=(
"1.11"
"1.12"
"1.13"
"1.14"
"1.15"
"1.16"
"1.17"
"1.18"
"1.19"
"1.20"
"1.21"
"1.22"
"1.23"
"1.24"
"1.25"
)
# Default values
PARALLEL=true
VERBOSE=false
OUTPUT_DIR="test-results"
DOCKER_TIMEOUT="10m"
usage() {
cat << EOF
Usage: $0 [OPTIONS] [GO_VERSIONS...]
Test go-toml across multiple Go versions using Docker containers.
The script reports the lowest continuous supported Go version (where all subsequent
versions pass) and only exits with non-zero status if either of the two most recent
Go versions fail, indicating immediate attention is needed.
Note: For Go versions < 1.21, the script automatically updates go.mod to match the
target version, but older versions may still fail due to missing standard library
features (e.g., the 'slices' package introduced in Go 1.21).
OPTIONS:
-h, --help Show this help message
-s, --sequential Run tests sequentially instead of in parallel
-v, --verbose Enable verbose output
-o, --output DIR Output directory for test results (default: test-results)
-t, --timeout TIME Docker timeout for each test (default: 10m)
--list List available Go versions and exit
ARGUMENTS:
GO_VERSIONS Specific Go versions to test (default: all supported versions)
Examples: 1.21 1.22 1.23
EXAMPLES:
$0 # Test all Go versions in parallel
$0 --sequential # Test all Go versions sequentially
$0 1.21 1.22 1.23 # Test specific versions
$0 --verbose --output ./results 1.24 1.25 # Verbose output to custom directory
EXIT CODES:
0 Recent Go versions pass (good compatibility)
1 Recent Go versions fail (needs attention) or script error
EOF
}
log() {
echo -e "${BLUE}[$(date +'%H:%M:%S')]${NC} $*" >&2
}
log_success() {
echo -e "${GREEN}[$(date +'%H:%M:%S')] ✓${NC} $*" >&2
}
log_error() {
echo -e "${RED}[$(date +'%H:%M:%S')] ✗${NC} $*" >&2
}
log_warning() {
echo -e "${YELLOW}[$(date +'%H:%M:%S')] ⚠${NC} $*" >&2
}
# Parse command line arguments
while [[ $# -gt 0 ]]; do
case $1 in
-h|--help)
usage
exit 0
;;
-s|--sequential)
PARALLEL=false
shift
;;
-v|--verbose)
VERBOSE=true
shift
;;
-o|--output)
OUTPUT_DIR="$2"
shift 2
;;
-t|--timeout)
DOCKER_TIMEOUT="$2"
shift 2
;;
--list)
echo "Available Go versions:"
printf '%s\n' "${GO_VERSIONS[@]}"
exit 0
;;
-*)
echo "Unknown option: $1" >&2
usage
exit 1
;;
*)
# Remaining arguments are Go versions
break
;;
esac
done
# If specific versions provided, use those instead of defaults
if [[ $# -gt 0 ]]; then
GO_VERSIONS=("$@")
fi
# Validate Go versions
for version in "${GO_VERSIONS[@]}"; do
if ! [[ "$version" =~ ^1\.(1[1-9]|2[0-5])$ ]]; then
log_error "Invalid Go version: $version. Supported versions: 1.11-1.25"
exit 1
fi
done
# Check if Docker is available
if ! command -v docker &> /dev/null; then
log_error "Docker is required but not installed or not in PATH"
exit 1
fi
# Check if Docker daemon is running
if ! docker info &> /dev/null; then
log_error "Docker daemon is not running"
exit 1
fi
# Create output directory
mkdir -p "$OUTPUT_DIR"
# Function to test a single Go version
test_go_version() {
local go_version="$1"
local container_name="go-toml-test-${go_version}"
local result_file="${OUTPUT_DIR}/go-${go_version}.txt"
local dockerfile_content
log "Testing Go $go_version..."
# Create a temporary Dockerfile for this version
# For Go versions < 1.21, we need to update go.mod to match the Go version
local needs_go_mod_update=false
if [[ $(echo "$go_version 1.21" | tr ' ' '\n' | sort -V | head -n1) == "$go_version" && "$go_version" != "1.21" ]]; then
needs_go_mod_update=true
fi
dockerfile_content="FROM golang:${go_version}-alpine
# Install git (required for go mod)
RUN apk add --no-cache git
# Set working directory
WORKDIR /app
# Copy source code
COPY . ."
# Add go.mod update step for older Go versions
if [[ "$needs_go_mod_update" == true ]]; then
dockerfile_content="$dockerfile_content
# Update go.mod to match Go version (required for Go < 1.21)
RUN if [ -f go.mod ]; then sed -i 's/^go [0-9]\\+\\.[0-9]\\+\\(\\.[0-9]\\+\\)\\?/go $go_version/' go.mod; fi
# Note: Go versions < 1.21 may fail due to missing standard library packages (e.g., slices)
# This is expected for projects that use Go 1.21+ features"
fi
dockerfile_content="$dockerfile_content
# Run tests
CMD [\"sh\", \"-c\", \"go version && echo '--- Running go test ./... ---' && go test ./...\"]"
# Create temporary directory for this test
local temp_dir
temp_dir=$(mktemp -d)
# Copy source to temp directory (excluding test results and git)
rsync -a --exclude="$OUTPUT_DIR" --exclude=".git" --exclude="*.test" . "$temp_dir/"
# Create Dockerfile in temp directory
echo "$dockerfile_content" > "$temp_dir/Dockerfile"
# Build and run container
local exit_code=0
local output
if $VERBOSE; then
log "Building Docker image for Go $go_version..."
fi
# Capture both stdout and stderr, and the exit code
if output=$(cd "$temp_dir" && timeout "$DOCKER_TIMEOUT" docker build -t "$container_name" . 2>&1 && \
timeout "$DOCKER_TIMEOUT" docker run --rm "$container_name" 2>&1); then
log_success "Go $go_version: PASSED"
echo "PASSED" > "${result_file}.status"
else
exit_code=$?
log_error "Go $go_version: FAILED (exit code: $exit_code)"
echo "FAILED" > "${result_file}.status"
fi
# Save full output
echo "$output" > "$result_file"
# Clean up
docker rmi "$container_name" &> /dev/null || true
rm -rf "$temp_dir"
if $VERBOSE; then
echo "--- Go $go_version output ---"
echo "$output"
echo "--- End Go $go_version output ---"
fi
return $exit_code
}
# Function to run tests in parallel
run_parallel() {
local pids=()
local failed_versions=()
log "Starting parallel tests for ${#GO_VERSIONS[@]} Go versions..."
# Start all tests in background
for version in "${GO_VERSIONS[@]}"; do
test_go_version "$version" &
pids+=($!)
done
# Wait for all tests to complete
for i in "${!pids[@]}"; do
local pid=${pids[$i]}
local version=${GO_VERSIONS[$i]}
if ! wait $pid; then
failed_versions+=("$version")
fi
done
return ${#failed_versions[@]}
}
# Function to run tests sequentially
run_sequential() {
local failed_versions=()
log "Starting sequential tests for ${#GO_VERSIONS[@]} Go versions..."
for version in "${GO_VERSIONS[@]}"; do
if ! test_go_version "$version"; then
failed_versions+=("$version")
fi
done
return ${#failed_versions[@]}
}
# Main execution
main() {
local start_time
start_time=$(date +%s)
log "Starting Go version compatibility tests..."
log "Testing versions: ${GO_VERSIONS[*]}"
log "Output directory: $OUTPUT_DIR"
log "Parallel execution: $PARALLEL"
local failed_count
if $PARALLEL; then
run_parallel
failed_count=$?
else
run_sequential
failed_count=$?
fi
local end_time
end_time=$(date +%s)
local duration=$((end_time - start_time))
# Collect results for display
local passed_versions=()
local failed_versions=()
local unknown_versions=()
local passed_count=0
for version in "${GO_VERSIONS[@]}"; do
local status_file="${OUTPUT_DIR}/go-${version}.txt.status"
if [[ -f "$status_file" ]]; then
local status
status=$(cat "$status_file")
if [[ "$status" == "PASSED" ]]; then
passed_versions+=("$version")
((passed_count++))
else
failed_versions+=("$version")
fi
else
unknown_versions+=("$version")
fi
done
# Generate summary report
local summary_file="${OUTPUT_DIR}/summary.txt"
{
echo "Go Version Compatibility Test Summary"
echo "====================================="
echo "Date: $(date)"
echo "Duration: ${duration}s"
echo "Parallel: $PARALLEL"
echo ""
echo "Results:"
for version in "${GO_VERSIONS[@]}"; do
local status_file="${OUTPUT_DIR}/go-${version}.txt.status"
if [[ -f "$status_file" ]]; then
local status
status=$(cat "$status_file")
if [[ "$status" == "PASSED" ]]; then
echo " Go $version: ✓ PASSED"
else
echo " Go $version: ✗ FAILED"
fi
else
echo " Go $version: ? UNKNOWN (no status file)"
fi
done
echo ""
echo "Summary: $passed_count/${#GO_VERSIONS[@]} versions passed"
if [[ $failed_count -gt 0 ]]; then
echo ""
echo "Failed versions details:"
for version in "${failed_versions[@]}"; do
echo ""
echo "--- Go $version (FAILED) ---"
local result_file="${OUTPUT_DIR}/go-${version}.txt"
if [[ -f "$result_file" ]]; then
tail -n 30 "$result_file"
fi
done
fi
} > "$summary_file"
# Find lowest continuous supported version and check recent versions
local lowest_continuous_version=""
local recent_versions_failed=false
# Sort versions to ensure proper order
local sorted_versions=()
for version in "${GO_VERSIONS[@]}"; do
sorted_versions+=("$version")
done
# Sort versions numerically (1.11, 1.12, ..., 1.25)
IFS=$'\n' sorted_versions=($(sort -V <<< "${sorted_versions[*]}"))
# Find lowest continuous supported version (all versions from this point onwards pass)
for version in "${sorted_versions[@]}"; do
local status_file="${OUTPUT_DIR}/go-${version}.txt.status"
local all_subsequent_pass=true
# Check if this version and all subsequent versions pass
local found_current=false
for check_version in "${sorted_versions[@]}"; do
if [[ "$check_version" == "$version" ]]; then
found_current=true
fi
if [[ "$found_current" == true ]]; then
local check_status_file="${OUTPUT_DIR}/go-${check_version}.txt.status"
if [[ -f "$check_status_file" ]]; then
local status
status=$(cat "$check_status_file")
if [[ "$status" != "PASSED" ]]; then
all_subsequent_pass=false
break
fi
else
all_subsequent_pass=false
break
fi
fi
done
if [[ "$all_subsequent_pass" == true ]]; then
lowest_continuous_version="$version"
break
fi
done
# Check if the two most recent versions failed
local num_versions=${#sorted_versions[@]}
if [[ $num_versions -ge 2 ]]; then
local second_recent="${sorted_versions[$((num_versions-2))]}"
local most_recent="${sorted_versions[$((num_versions-1))]}"
local second_recent_status_file="${OUTPUT_DIR}/go-${second_recent}.txt.status"
local most_recent_status_file="${OUTPUT_DIR}/go-${most_recent}.txt.status"
local second_recent_failed=false
local most_recent_failed=false
if [[ -f "$second_recent_status_file" ]]; then
local status
status=$(cat "$second_recent_status_file")
if [[ "$status" != "PASSED" ]]; then
second_recent_failed=true
fi
else
second_recent_failed=true
fi
if [[ -f "$most_recent_status_file" ]]; then
local status
status=$(cat "$most_recent_status_file")
if [[ "$status" != "PASSED" ]]; then
most_recent_failed=true
fi
else
most_recent_failed=true
fi
if [[ "$second_recent_failed" == true || "$most_recent_failed" == true ]]; then
recent_versions_failed=true
fi
elif [[ $num_versions -eq 1 ]]; then
# Only one version tested, check if it's the most recent and failed
local only_version="${sorted_versions[0]}"
local only_status_file="${OUTPUT_DIR}/go-${only_version}.txt.status"
if [[ -f "$only_status_file" ]]; then
local status
status=$(cat "$only_status_file")
if [[ "$status" != "PASSED" ]]; then
recent_versions_failed=true
fi
else
recent_versions_failed=true
fi
fi
# Display summary
echo ""
log "Test completed in ${duration}s"
log "Summary report: $summary_file"
echo ""
echo "========================================"
echo " FINAL RESULTS"
echo "========================================"
echo ""
# Display passed versions
if [[ ${#passed_versions[@]} -gt 0 ]]; then
log_success "PASSED (${#passed_versions[@]}/${#GO_VERSIONS[@]}):"
# Sort passed versions for display
local sorted_passed=()
for version in "${sorted_versions[@]}"; do
for passed_version in "${passed_versions[@]}"; do
if [[ "$version" == "$passed_version" ]]; then
sorted_passed+=("$version")
break
fi
done
done
for version in "${sorted_passed[@]}"; do
echo -e " ${GREEN}${NC} Go $version"
done
echo ""
fi
# Display failed versions
if [[ ${#failed_versions[@]} -gt 0 ]]; then
log_error "FAILED (${#failed_versions[@]}/${#GO_VERSIONS[@]}):"
# Sort failed versions for display
local sorted_failed=()
for version in "${sorted_versions[@]}"; do
for failed_version in "${failed_versions[@]}"; do
if [[ "$version" == "$failed_version" ]]; then
sorted_failed+=("$version")
break
fi
done
done
for version in "${sorted_failed[@]}"; do
echo -e " ${RED}${NC} Go $version"
done
echo ""
# Show failure details
echo "========================================"
echo " FAILURE DETAILS"
echo "========================================"
echo ""
for version in "${sorted_failed[@]}"; do
echo -e "${RED}--- Go $version FAILURE LOGS (last 30 lines) ---${NC}"
local result_file="${OUTPUT_DIR}/go-${version}.txt"
if [[ -f "$result_file" ]]; then
tail -n 30 "$result_file" | sed 's/^/ /'
else
echo " No log file found: $result_file"
fi
echo ""
done
fi
# Display unknown versions
if [[ ${#unknown_versions[@]} -gt 0 ]]; then
log_warning "UNKNOWN (${#unknown_versions[@]}/${#GO_VERSIONS[@]}):"
for version in "${unknown_versions[@]}"; do
echo -e " ${YELLOW}?${NC} Go $version (no status file)"
done
echo ""
fi
echo "========================================"
echo " COMPATIBILITY SUMMARY"
echo "========================================"
echo ""
if [[ -n "$lowest_continuous_version" ]]; then
log_success "Lowest continuous supported version: Go $lowest_continuous_version"
echo " (All versions from Go $lowest_continuous_version onwards pass)"
else
log_error "No continuous version support found"
echo " (No version has all subsequent versions passing)"
fi
echo ""
echo "========================================"
echo "Full detailed logs available in: $OUTPUT_DIR"
echo "========================================"
# Determine exit code based on recent versions
if [[ "$recent_versions_failed" == true ]]; then
log_error "OVERALL RESULT: Recent Go versions failed - this needs attention!"
if [[ -n "$lowest_continuous_version" ]]; then
echo "Note: Continuous support starts from Go $lowest_continuous_version"
fi
exit 1
else
log_success "OVERALL RESULT: Recent Go versions pass - compatibility looks good!"
if [[ -n "$lowest_continuous_version" ]]; then
echo "Continuous support starts from Go $lowest_continuous_version"
fi
exit 0
fi
}
# Trap to clean up on exit
cleanup() {
# Kill any remaining background processes
jobs -p | xargs -r kill 2>/dev/null || true
# Clean up any remaining Docker containers
docker ps -q --filter "name=go-toml-test-" | xargs -r docker stop 2>/dev/null || true
docker images -q --filter "reference=go-toml-test-*" | xargs -r docker rmi 2>/dev/null || true
}
trap cleanup EXIT
# Run main function
main
+7 -6
View File
@@ -1,4 +1,5 @@
//go:generate go run ./cmd/tomltestgen/main.go -o toml_testgen_test.go
//go:generate go run github.com/toml-lang/toml-test/cmd/toml-test@v1.6.0 -copy ./tests
//go:generate go run ./cmd/tomltestgen/main.go -r v1.6.0 -o toml_testgen_test.go
// This is a support file for toml_testgen_test.go
package toml_test
@@ -8,8 +9,8 @@ import (
"testing"
"github.com/pelletier/go-toml/v2"
"github.com/pelletier/go-toml/v2/internal/assert"
"github.com/pelletier/go-toml/v2/internal/testsuite"
"github.com/stretchr/testify/require"
)
func testgenInvalid(t *testing.T, input string) {
@@ -44,15 +45,15 @@ func testgenValid(t *testing.T, input string, jsonRef string) {
t.Fatalf("failed parsing toml: %s", err)
}
j, err := testsuite.ValueToTaggedJSON(doc)
require.NoError(t, err)
assert.NoError(t, err)
var ref interface{}
err = json.Unmarshal([]byte(jsonRef), &ref)
require.NoError(t, err)
assert.NoError(t, err)
var actual interface{}
err = json.Unmarshal([]byte(j), &actual)
require.NoError(t, err)
err = json.Unmarshal(j, &actual)
assert.NoError(t, err)
testsuite.CmpJSON(t, "", ref, actual)
}
+1634 -433
View File
File diff suppressed because it is too large Load Diff
+188 -37
View File
@@ -5,14 +5,13 @@ import (
"errors"
"fmt"
"io"
"io/ioutil"
"math"
"reflect"
"strconv"
"strings"
"sync/atomic"
"time"
"github.com/pelletier/go-toml/v2/internal/danger"
"github.com/pelletier/go-toml/v2/internal/tracker"
"github.com/pelletier/go-toml/v2/unstable"
)
@@ -21,10 +20,8 @@ import (
//
// It is a shortcut for Decoder.Decode() with the default options.
func Unmarshal(data []byte, v interface{}) error {
p := unstable.Parser{}
p.Reset(data)
d := decoder{p: &p}
d := decoder{}
d.p.Reset(data)
return d.FromParser(v)
}
@@ -35,6 +32,9 @@ type Decoder struct {
// global settings
strict bool
// toggles unmarshaler interface
unmarshalerInterface bool
}
// NewDecoder creates a new Decoder that will read from r.
@@ -54,6 +54,24 @@ func (d *Decoder) DisallowUnknownFields() *Decoder {
return d
}
// EnableUnmarshalerInterface allows to enable unmarshaler interface.
//
// With this feature enabled, types implementing the unstable/Unmarshaler
// interface can be decoded from any structure of the document. It allows types
// that don't have a straightforward TOML representation to provide their own
// decoding logic.
//
// Currently, types can only decode from a single value. Tables and array tables
// are not supported.
//
// *Unstable:* This method does not follow the compatibility guarantees of
// semver. It can be changed or removed without a new major version being
// issued.
func (d *Decoder) EnableUnmarshalerInterface() *Decoder {
d.unmarshalerInterface = true
return d
}
// Decode the whole content of r into v.
//
// By default, values in the document that don't exist in the target Go value
@@ -96,26 +114,25 @@ func (d *Decoder) DisallowUnknownFields() *Decoder {
// Inline Table -> same as Table
// Array of Tables -> same as Array and Table
func (d *Decoder) Decode(v interface{}) error {
b, err := ioutil.ReadAll(d.r)
b, err := io.ReadAll(d.r)
if err != nil {
return fmt.Errorf("toml: %w", err)
}
p := unstable.Parser{}
p.Reset(b)
dec := decoder{
p: &p,
strict: strict{
Enabled: d.strict,
},
unmarshalerInterface: d.unmarshalerInterface,
}
dec.p.Reset(b)
return dec.FromParser(v)
}
type decoder struct {
// Which parser instance in use for this decoding session.
p *unstable.Parser
p unstable.Parser
// Flag indicating that the current expression is stashed.
// If set to true, calling nextExpr will not actually pull a new expression
@@ -127,6 +144,10 @@ type decoder struct {
// need to be skipped.
skipUntilTable bool
// Flag indicating that the current array/slice table should be cleared because
// it is the first encounter of an array table.
clearArrayTable bool
// Tracks position in Go arrays.
// This is used when decoding [[array tables]] into Go arrays. Given array
// tables are separate TOML expression, we need to keep track of where we
@@ -139,6 +160,9 @@ type decoder struct {
// Strict mode
strict strict
// Flag that enables/disables unmarshaler interface.
unmarshalerInterface bool
// Current context for the error.
errorContext *errorContext
}
@@ -149,12 +173,16 @@ type errorContext struct {
}
func (d *decoder) typeMismatchError(toml string, target reflect.Type) error {
return fmt.Errorf("toml: %s", d.typeMismatchString(toml, target))
}
func (d *decoder) typeMismatchString(toml string, target reflect.Type) string {
if d.errorContext != nil && d.errorContext.Struct != nil {
ctx := d.errorContext
f := ctx.Struct.FieldByIndex(ctx.Field)
return fmt.Errorf("toml: cannot decode TOML %s into struct field %s.%s of type %s", toml, ctx.Struct, f.Name, f.Type)
return fmt.Sprintf("cannot decode TOML %s into struct field %s.%s of type %s", toml, ctx.Struct, f.Name, f.Type)
}
return fmt.Errorf("toml: cannot decode TOML %s into a Go value of type %s", toml, target)
return fmt.Sprintf("cannot decode TOML %s into a Go value of type %s", toml, target)
}
func (d *decoder) expr() *unstable.Node {
@@ -242,9 +270,10 @@ Rules for the unmarshal code:
func (d *decoder) handleRootExpression(expr *unstable.Node, v reflect.Value) error {
var x reflect.Value
var err error
var first bool // used for to clear array tables on first use
if !(d.skipUntilTable && expr.Kind == unstable.KeyValue) {
err = d.seen.CheckExpression(expr)
first, err = d.seen.CheckExpression(expr)
if err != nil {
return err
}
@@ -263,6 +292,7 @@ func (d *decoder) handleRootExpression(expr *unstable.Node, v reflect.Value) err
case unstable.ArrayTable:
d.skipUntilTable = false
d.strict.EnterArrayTable(expr)
d.clearArrayTable = first
x, err = d.handleArrayTable(expr.Key(), v)
default:
panic(fmt.Errorf("parser should not permit expression of kind %s at document root", expr.Kind))
@@ -303,6 +333,10 @@ func (d *decoder) handleArrayTableCollectionLast(key unstable.Iterator, v reflec
reflect.Copy(nelem, elem)
elem = nelem
}
if d.clearArrayTable && elem.Len() > 0 {
elem.SetLen(0)
d.clearArrayTable = false
}
}
return d.handleArrayTableCollectionLast(key, elem)
case reflect.Ptr:
@@ -321,6 +355,10 @@ func (d *decoder) handleArrayTableCollectionLast(key unstable.Iterator, v reflec
return v, nil
case reflect.Slice:
if d.clearArrayTable && v.Len() > 0 {
v.SetLen(0)
d.clearArrayTable = false
}
elemType := v.Type().Elem()
var elem reflect.Value
if elemType.Kind() == reflect.Interface {
@@ -377,15 +415,39 @@ func (d *decoder) handleArrayTableCollection(key unstable.Iterator, v reflect.Va
return v, nil
case reflect.Slice:
elem := v.Index(v.Len() - 1)
// Create a new element when the slice is empty; otherwise operate on
// the last element.
var (
elem reflect.Value
created bool
)
if v.Len() == 0 {
created = true
elemType := v.Type().Elem()
if elemType.Kind() == reflect.Interface {
elem = makeMapStringInterface()
} else {
elem = reflect.New(elemType).Elem()
}
} else {
elem = v.Index(v.Len() - 1)
}
x, err := d.handleArrayTable(key, elem)
if err != nil || d.skipUntilTable {
return reflect.Value{}, err
}
if x.IsValid() {
elem.Set(x)
if created {
elem = x
} else {
elem.Set(x)
}
}
if created {
return reflect.Append(v, elem), nil
}
return v, err
case reflect.Array:
idx := d.arrayIndex(false, v)
@@ -417,7 +479,10 @@ func (d *decoder) handleKeyPart(key unstable.Iterator, v reflect.Value, nextFn h
vt := v.Type()
// Create the key for the map element. Convert to key type.
mk := reflect.ValueOf(string(key.Node().Data)).Convert(vt.Key())
mk, err := d.keyFromData(vt.Key(), key.Node().Data)
if err != nil {
return reflect.Value{}, err
}
// If the map does not exist, create it.
if v.IsNil() {
@@ -569,7 +634,7 @@ func (d *decoder) handleKeyValues(v reflect.Value) (reflect.Value, error) {
break
}
err := d.seen.CheckExpression(expr)
_, err := d.seen.CheckExpression(expr)
if err != nil {
return reflect.Value{}, err
}
@@ -627,6 +692,14 @@ func (d *decoder) handleValue(value *unstable.Node, v reflect.Value) error {
v = initAndDereferencePointer(v)
}
if d.unmarshalerInterface {
if v.CanAddr() && v.Addr().CanInterface() {
if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok {
return outi.UnmarshalTOML(value)
}
}
}
ok, err := d.tryTextUnmarshaler(value, v)
if ok || err != nil {
return err
@@ -960,7 +1033,7 @@ func (d *decoder) unmarshalInteger(value *unstable.Node, v reflect.Value) error
case reflect.Interface:
r = reflect.ValueOf(i)
default:
return d.typeMismatchError("integer", v.Type())
return unstable.NewParserError(d.p.Raw(value.Raw), "%s", d.typeMismatchString("integer", v.Type()))
}
if !r.Type().AssignableTo(v.Type()) {
@@ -979,7 +1052,7 @@ func (d *decoder) unmarshalString(value *unstable.Node, v reflect.Value) error {
case reflect.Interface:
v.Set(reflect.ValueOf(string(value.Data)))
default:
return unstable.NewParserError(d.p.Raw(value.Raw), "cannot store TOML string into a Go %s", v.Kind())
return unstable.NewParserError(d.p.Raw(value.Raw), "%s", d.typeMismatchString("string", v.Type()))
}
return nil
@@ -1009,6 +1082,58 @@ func (d *decoder) handleKeyValueInner(key unstable.Iterator, value *unstable.Nod
return reflect.Value{}, d.handleValue(value, v)
}
func (d *decoder) keyFromData(keyType reflect.Type, data []byte) (reflect.Value, error) {
switch {
case stringType.AssignableTo(keyType):
return reflect.ValueOf(string(data)), nil
case stringType.ConvertibleTo(keyType):
return reflect.ValueOf(string(data)).Convert(keyType), nil
case keyType.Implements(textUnmarshalerType):
mk := reflect.New(keyType.Elem())
if err := mk.Interface().(encoding.TextUnmarshaler).UnmarshalText(data); err != nil {
return reflect.Value{}, fmt.Errorf("toml: error unmarshalling key type %s from text: %w", stringType, err)
}
return mk, nil
case reflect.PointerTo(keyType).Implements(textUnmarshalerType):
mk := reflect.New(keyType)
if err := mk.Interface().(encoding.TextUnmarshaler).UnmarshalText(data); err != nil {
return reflect.Value{}, fmt.Errorf("toml: error unmarshalling key type %s from text: %w", stringType, err)
}
return mk.Elem(), nil
case keyType.Kind() == reflect.Int || keyType.Kind() == reflect.Int8 || keyType.Kind() == reflect.Int16 || keyType.Kind() == reflect.Int32 || keyType.Kind() == reflect.Int64:
key, err := strconv.ParseInt(string(data), 10, 64)
if err != nil {
return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from integer: %w", stringType, err)
}
return reflect.ValueOf(key).Convert(keyType), nil
case keyType.Kind() == reflect.Uint || keyType.Kind() == reflect.Uint8 || keyType.Kind() == reflect.Uint16 || keyType.Kind() == reflect.Uint32 || keyType.Kind() == reflect.Uint64:
key, err := strconv.ParseUint(string(data), 10, 64)
if err != nil {
return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from unsigned integer: %w", stringType, err)
}
return reflect.ValueOf(key).Convert(keyType), nil
case keyType.Kind() == reflect.Float32:
key, err := strconv.ParseFloat(string(data), 32)
if err != nil {
return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from float: %w", stringType, err)
}
return reflect.ValueOf(float32(key)), nil
case keyType.Kind() == reflect.Float64:
key, err := strconv.ParseFloat(string(data), 64)
if err != nil {
return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from float: %w", stringType, err)
}
return reflect.ValueOf(float64(key)), nil
}
return reflect.Value{}, fmt.Errorf("toml: cannot convert map key of type %s to expected type %s", stringType, keyType)
}
func (d *decoder) handleKeyValuePart(key unstable.Iterator, value *unstable.Node, v reflect.Value) (reflect.Value, error) {
// contains the replacement for v
var rv reflect.Value
@@ -1019,16 +1144,9 @@ func (d *decoder) handleKeyValuePart(key unstable.Iterator, value *unstable.Node
case reflect.Map:
vt := v.Type()
mk := reflect.ValueOf(string(key.Node().Data))
mkt := stringType
keyType := vt.Key()
if !mkt.AssignableTo(keyType) {
if !mkt.ConvertibleTo(keyType) {
return reflect.Value{}, fmt.Errorf("toml: cannot convert map key of type %s to expected type %s", mkt, keyType)
}
mk = mk.Convert(keyType)
mk, err := d.keyFromData(vt.Key(), key.Node().Data)
if err != nil {
return reflect.Value{}, err
}
// If the map does not exist, create it.
@@ -1059,6 +1177,17 @@ func (d *decoder) handleKeyValuePart(key unstable.Iterator, value *unstable.Node
case reflect.Struct:
path, found := structFieldPath(v, string(key.Node().Data))
if !found {
// If no matching struct field is found but the target implements the
// unstable.Unmarshaler interface (and it is enabled), delegate the
// decoding of this value to the custom unmarshaler.
if d.unmarshalerInterface {
if v.CanAddr() && v.Addr().CanInterface() {
if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok {
return reflect.Value{}, outi.UnmarshalTOML(value)
}
}
}
// Otherwise, keep previous behavior and skip until the next table.
d.skipUntilTable = true
break
}
@@ -1071,6 +1200,19 @@ func (d *decoder) handleKeyValuePart(key unstable.Iterator, value *unstable.Node
d.errorContext.Field = path
f := fieldByIndex(v, path)
if !f.CanAddr() {
// If the field is not addressable, need to take a slower path and
// make a copy of the struct itself to a new location.
nvp := reflect.New(v.Type())
nvp.Elem().Set(v)
v = nvp.Elem()
_, err := d.handleKeyValuePart(key, value, v)
if err != nil {
return reflect.Value{}, err
}
return nvp.Elem(), nil
}
x, err := d.handleKeyValueInner(key, value, f)
if err != nil {
return reflect.Value{}, err
@@ -1136,10 +1278,10 @@ func initAndDereferencePointer(v reflect.Value) reflect.Value {
// Same as reflect.Value.FieldByIndex, but creates pointers if needed.
func fieldByIndex(v reflect.Value, path []int) reflect.Value {
for i, x := range path {
for _, x := range path {
v = v.Field(x)
if i < len(path)-1 && v.Kind() == reflect.Ptr {
if v.Kind() == reflect.Ptr {
if v.IsNil() {
v.Set(reflect.New(v.Type().Elem()))
}
@@ -1151,13 +1293,22 @@ func fieldByIndex(v reflect.Value, path []int) reflect.Value {
type fieldPathsMap = map[string][]int
var globalFieldPathsCache atomic.Value // map[danger.TypeID]fieldPathsMap
var globalFieldPathsCache atomic.Value // map[uintptr]fieldPathsMap
func structFieldPath(v reflect.Value, name string) ([]int, bool) {
t := v.Type()
// reflect.Type is an interface. We want to use the address of the underlying
// rtype as the key.
// This avoids using the interface as map key, which is slower.
//
// In the future this should be replaced by t.Pointer() if it becomes available.
//
// v.Type() returns a reflect.Type interface.
// reflect.ValueOf(t).Pointer() returns the address of the rtype.
tid := reflect.ValueOf(t).Pointer()
cache, _ := globalFieldPathsCache.Load().(map[danger.TypeID]fieldPathsMap)
fieldPaths, ok := cache[danger.MakeTypeID(t)]
cache, _ := globalFieldPathsCache.Load().(map[uintptr]fieldPathsMap)
fieldPaths, ok := cache[tid]
if !ok {
fieldPaths = map[string][]int{}
@@ -1168,8 +1319,8 @@ func structFieldPath(v reflect.Value, name string) ([]int, bool) {
fieldPaths[strings.ToLower(name)] = path
})
newCache := make(map[danger.TypeID]fieldPathsMap, len(cache)+1)
newCache[danger.MakeTypeID(t)] = fieldPaths
newCache := make(map[uintptr]fieldPathsMap, len(cache)+1)
newCache[tid] = fieldPaths
for k, v := range cache {
newCache[k] = v
}
+1019 -99
View File
File diff suppressed because it is too large Load Diff
+7 -22
View File
@@ -2,9 +2,6 @@ package unstable
import (
"fmt"
"unsafe"
"github.com/pelletier/go-toml/v2/internal/danger"
)
// Iterator over a sequence of nodes.
@@ -37,7 +34,7 @@ func (c *Iterator) Next() bool {
// IsLast returns true if the current node of the iterator is the last
// one. Subsequent calls to Next() will return false.
func (c *Iterator) IsLast() bool {
return c.node.next == 0
return c.node.next == nil
}
// Node returns a pointer to the node pointed at by the iterator.
@@ -65,11 +62,9 @@ type Node struct {
Raw Range // Raw bytes from the input.
Data []byte // Node value (either allocated or referencing the input).
// References to other nodes, as offsets in the backing array
// from this node. References can go backward, so those can be
// negative.
next int // 0 if last element
child int // 0 if no child
// References to other nodes.
next *Node // nil if last element
child *Node // nil if no child
}
// Range of bytes in the document.
@@ -80,24 +75,14 @@ type Range struct {
// Next returns a pointer to the next node, or nil if there is no next node.
func (n *Node) Next() *Node {
if n.next == 0 {
return nil
}
ptr := unsafe.Pointer(n)
size := unsafe.Sizeof(Node{})
return (*Node)(danger.Stride(ptr, size, n.next))
return n.next
}
// Child returns a pointer to the first child node of this node. Other children
// can be accessed calling Next on the first child. Returns an nil if this Node
// can be accessed calling Next on the first child. Returns nil if this Node
// has no child.
func (n *Node) Child() *Node {
if n.child == 0 {
return nil
}
ptr := unsafe.Pointer(n)
size := unsafe.Sizeof(Node{})
return (*Node)(danger.Stride(ptr, size, n.child))
return n.child
}
// Valid returns true if the node's kind is set (not to Invalid).
+79 -29
View File
@@ -4,68 +4,118 @@ package unstable
//
// It is immutable once constructed with Builder.
type root struct {
nodes []Node
first *Node
}
// Iterator over the top level nodes.
func (r *root) Iterator() Iterator {
it := Iterator{}
if len(r.nodes) > 0 {
it.node = &r.nodes[0]
}
return it
return Iterator{node: r.first}
}
func (r *root) at(idx reference) *Node {
return &r.nodes[idx]
type reference struct {
*Node
}
type reference int
const invalidReference reference = -1
var invalidReference = reference{}
func (r reference) Valid() bool {
return r != invalidReference
return r.Node != nil
}
type builder struct {
tree root
lastIdx int
// chunks of nodes. Pointers to nodes are stable because we only append
// to the last chunk, and chunks are allocated with fixed capacity.
chunks [][]Node
// current chunk index
chunkIdx int
// root node of the tree
root root
// last pushed node (for chaining)
last *Node
}
const initialChunkSize = 16
const maxChunkSize = 2048
func (b *builder) Tree() *root {
return &b.tree
return &b.root
}
func (b *builder) NodeAt(ref reference) *Node {
return b.tree.at(ref)
return ref.Node
}
func (b *builder) Reset() {
b.tree.nodes = b.tree.nodes[:0]
b.lastIdx = 0
b.chunkIdx = 0
for i := range b.chunks {
b.chunks[i] = b.chunks[i][:0]
}
b.root.first = nil
b.last = nil
}
func (b *builder) ensureCapacity() {
if b.chunkIdx >= len(b.chunks) {
size := initialChunkSize
if len(b.chunks) > 0 {
lastCap := cap(b.chunks[len(b.chunks)-1])
size = lastCap * 2
if size > maxChunkSize {
size = maxChunkSize
}
}
b.chunks = append(b.chunks, make([]Node, 0, size))
}
if len(b.chunks[b.chunkIdx]) == cap(b.chunks[b.chunkIdx]) {
b.chunkIdx++
if b.chunkIdx >= len(b.chunks) {
size := initialChunkSize
if len(b.chunks) > 0 {
lastCap := cap(b.chunks[len(b.chunks)-1])
size = lastCap * 2
if size > maxChunkSize {
size = maxChunkSize
}
}
b.chunks = append(b.chunks, make([]Node, 0, size))
}
}
}
func (b *builder) push(n Node) *Node {
b.ensureCapacity()
chunk := &b.chunks[b.chunkIdx]
*chunk = append(*chunk, n)
return &(*chunk)[len(*chunk)-1]
}
func (b *builder) Push(n Node) reference {
b.lastIdx = len(b.tree.nodes)
b.tree.nodes = append(b.tree.nodes, n)
return reference(b.lastIdx)
ptr := b.push(n)
if b.root.first == nil {
b.root.first = ptr
}
b.last = ptr
return reference{ptr}
}
func (b *builder) PushAndChain(n Node) reference {
newIdx := len(b.tree.nodes)
b.tree.nodes = append(b.tree.nodes, n)
if b.lastIdx >= 0 {
b.tree.nodes[b.lastIdx].next = newIdx - b.lastIdx
ptr := b.push(n)
if b.root.first == nil {
b.root.first = ptr
}
b.lastIdx = newIdx
return reference(b.lastIdx)
if b.last != nil {
b.last.next = ptr
}
b.last = ptr
return reference{ptr}
}
func (b *builder) AttachChild(parent reference, child reference) {
b.tree.nodes[parent].child = int(child) - int(parent)
parent.child = child.Node
}
func (b *builder) Chain(from reference, to reference) {
b.tree.nodes[from].next = int(to) - int(from)
from.next = to.Node
}
+118 -22
View File
@@ -6,7 +6,6 @@ import (
"unicode"
"github.com/pelletier/go-toml/v2/internal/characters"
"github.com/pelletier/go-toml/v2/internal/danger"
)
// ParserError describes an error relative to the content of the document.
@@ -49,8 +48,6 @@ func NewParserError(highlight []byte, format string, args ...interface{}) error
// For performance reasons, go-toml doesn't make a copy of the input bytes to
// the parser. Make sure to copy all the bytes you need to outlive the slice
// given to the parser.
//
// The parser doesn't provide nodes for comments yet, nor for whitespace.
type Parser struct {
data []byte
builder builder
@@ -58,6 +55,8 @@ type Parser struct {
left []byte
err error
first bool
KeepComments bool
}
// Data returns the slice provided to the last call to Reset.
@@ -70,7 +69,7 @@ func (p *Parser) Data() []byte {
// panics.
func (p *Parser) Range(b []byte) Range {
return Range{
Offset: uint32(danger.SubsliceOffset(p.data, b)),
Offset: uint32(cap(p.data) - cap(b)),
Length: uint32(len(b)),
}
}
@@ -142,6 +141,44 @@ func (p *Parser) Error() error {
return p.err
}
// Position describes a position in the input.
type Position struct {
// Number of bytes from the beginning of the input.
Offset int
// Line number, starting at 1.
Line int
// Column number, starting at 1.
Column int
}
// Shape describes the position of a range in the input.
type Shape struct {
Start Position
End Position
}
func (p *Parser) position(b []byte) Position {
offset := cap(p.data) - cap(b)
lead := p.data[:offset]
return Position{
Offset: offset,
Line: bytes.Count(lead, []byte{'\n'}) + 1,
Column: len(lead) - bytes.LastIndex(lead, []byte{'\n'}),
}
}
// Shape returns the shape of the given range in the input. Will
// panic if the range is not a subslice of the input.
func (p *Parser) Shape(r Range) Shape {
raw := p.Raw(r)
return Shape{
Start: p.position(raw),
End: p.position(raw[r.Length:]),
}
}
func (p *Parser) parseNewline(b []byte) ([]byte, error) {
if b[0] == '\n' {
return b[1:], nil
@@ -155,6 +192,19 @@ func (p *Parser) parseNewline(b []byte) ([]byte, error) {
return nil, NewParserError(b[0:1], "expected newline but got %#U", b[0])
}
func (p *Parser) parseComment(b []byte) (reference, []byte, error) {
ref := invalidReference
data, rest, err := scanComment(b)
if p.KeepComments && err == nil {
ref = p.builder.Push(Node{
Kind: Comment,
Raw: p.Range(data),
Data: data,
})
}
return ref, rest, err
}
func (p *Parser) parseExpression(b []byte) (reference, []byte, error) {
// expression = ws [ comment ]
// expression =/ ws keyval ws [ comment ]
@@ -168,7 +218,7 @@ func (p *Parser) parseExpression(b []byte) (reference, []byte, error) {
}
if b[0] == '#' {
_, rest, err := scanComment(b)
ref, rest, err := p.parseComment(b)
return ref, rest, err
}
@@ -190,7 +240,10 @@ func (p *Parser) parseExpression(b []byte) (reference, []byte, error) {
b = p.parseWhitespace(b)
if len(b) > 0 && b[0] == '#' {
_, rest, err := scanComment(b)
cref, rest, err := p.parseComment(b)
if cref != invalidReference {
p.builder.Chain(ref, cref)
}
return ref, rest, err
}
@@ -471,17 +524,33 @@ func (p *Parser) parseValArray(b []byte) (reference, []byte, error) {
Kind: Array,
})
// First indicates whether the parser is looking for the first element
// (non-comment) of the array.
first := true
var lastChild reference
lastChild := invalidReference
addChild := func(valueRef reference) {
if lastChild == invalidReference {
p.builder.AttachChild(parent, valueRef)
} else {
p.builder.Chain(lastChild, valueRef)
}
lastChild = valueRef
}
var err error
for len(b) > 0 {
b, err = p.parseOptionalWhitespaceCommentNewline(b)
cref := invalidReference
cref, b, err = p.parseOptionalWhitespaceCommentNewline(b)
if err != nil {
return parent, nil, err
}
if cref != invalidReference {
addChild(cref)
}
if len(b) == 0 {
return parent, nil, NewParserError(arrayStart[:1], "array is incomplete")
}
@@ -496,10 +565,13 @@ func (p *Parser) parseValArray(b []byte) (reference, []byte, error) {
}
b = b[1:]
b, err = p.parseOptionalWhitespaceCommentNewline(b)
cref, b, err = p.parseOptionalWhitespaceCommentNewline(b)
if err != nil {
return parent, nil, err
}
if cref != invalidReference {
addChild(cref)
}
} else if !first {
return parent, nil, NewParserError(b[0:1], "array elements must be separated by commas")
}
@@ -515,17 +587,16 @@ func (p *Parser) parseValArray(b []byte) (reference, []byte, error) {
return parent, nil, err
}
if first {
p.builder.AttachChild(parent, valueRef)
} else {
p.builder.Chain(lastChild, valueRef)
}
lastChild = valueRef
addChild(valueRef)
b, err = p.parseOptionalWhitespaceCommentNewline(b)
cref, b, err = p.parseOptionalWhitespaceCommentNewline(b)
if err != nil {
return parent, nil, err
}
if cref != invalidReference {
addChild(cref)
}
first = false
}
@@ -534,15 +605,34 @@ func (p *Parser) parseValArray(b []byte) (reference, []byte, error) {
return parent, rest, err
}
func (p *Parser) parseOptionalWhitespaceCommentNewline(b []byte) ([]byte, error) {
func (p *Parser) parseOptionalWhitespaceCommentNewline(b []byte) (reference, []byte, error) {
rootCommentRef := invalidReference
latestCommentRef := invalidReference
addComment := func(ref reference) {
if rootCommentRef == invalidReference {
rootCommentRef = ref
} else if latestCommentRef == invalidReference {
p.builder.AttachChild(rootCommentRef, ref)
latestCommentRef = ref
} else {
p.builder.Chain(latestCommentRef, ref)
latestCommentRef = ref
}
}
for len(b) > 0 {
var err error
b = p.parseWhitespace(b)
if len(b) > 0 && b[0] == '#' {
_, b, err = scanComment(b)
var ref reference
ref, b, err = p.parseComment(b)
if err != nil {
return nil, err
return invalidReference, nil, err
}
if ref != invalidReference {
addComment(ref)
}
}
@@ -553,14 +643,14 @@ func (p *Parser) parseOptionalWhitespaceCommentNewline(b []byte) ([]byte, error)
if b[0] == '\n' || b[0] == '\r' {
b, err = p.parseNewline(b)
if err != nil {
return nil, err
return invalidReference, nil, err
}
} else {
break
}
}
return b, nil
return rootCommentRef, b, nil
}
func (p *Parser) parseMultilineLiteralString(b []byte) ([]byte, []byte, []byte, error) {
@@ -922,6 +1012,7 @@ func (p *Parser) parseIntOrFloatOrDateTime(b []byte) (reference, []byte, error)
return p.builder.Push(Node{
Kind: Float,
Data: b[:3],
Raw: p.Range(b[:3]),
}), b[3:], nil
case 'n':
if !scanFollowsNan(b) {
@@ -931,6 +1022,7 @@ func (p *Parser) parseIntOrFloatOrDateTime(b []byte) (reference, []byte, error)
return p.builder.Push(Node{
Kind: Float,
Data: b[:3],
Raw: p.Range(b[:3]),
}), b[3:], nil
case '+', '-':
return p.scanIntOrFloat(b)
@@ -983,7 +1075,7 @@ byteLoop:
}
case c == 'T' || c == 't' || c == ':' || c == '.':
hasTime = true
case c == '+' || c == '-' || c == 'Z' || c == 'z':
case c == '+' || c == 'Z' || c == 'z':
hasTz = true
case c == ' ':
if !seenSpace && i+1 < len(b) && isDigit(b[i+1]) {
@@ -1055,6 +1147,7 @@ func (p *Parser) scanIntOrFloat(b []byte) (reference, []byte, error) {
return p.builder.Push(Node{
Kind: Integer,
Data: b[:i],
Raw: p.Range(b[:i]),
}), b[i:], nil
}
@@ -1078,6 +1171,7 @@ func (p *Parser) scanIntOrFloat(b []byte) (reference, []byte, error) {
return p.builder.Push(Node{
Kind: Float,
Data: b[:i+3],
Raw: p.Range(b[:i+3]),
}), b[i+3:], nil
}
@@ -1089,6 +1183,7 @@ func (p *Parser) scanIntOrFloat(b []byte) (reference, []byte, error) {
return p.builder.Push(Node{
Kind: Float,
Data: b[:i+3],
Raw: p.Range(b[:i+3]),
}), b[i+3:], nil
}
@@ -1111,6 +1206,7 @@ func (p *Parser) scanIntOrFloat(b []byte) (reference, []byte, error) {
return p.builder.Push(Node{
Kind: kind,
Data: b[:i],
Raw: p.Range(b[:i]),
}), b[i:], nil
}
+166 -9
View File
@@ -6,7 +6,7 @@ import (
"strings"
"testing"
"github.com/stretchr/testify/require"
"github.com/pelletier/go-toml/v2/internal/assert"
)
func TestParser_AST_Numbers(t *testing.T) {
@@ -141,9 +141,9 @@ func TestParser_AST_Numbers(t *testing.T) {
p.NextExpression()
err := p.Error()
if e.err {
require.Error(t, err)
assert.Error(t, err)
} else {
require.NoError(t, err)
assert.NoError(t, err)
expected := astNode{
Kind: KeyValue,
@@ -168,8 +168,8 @@ type (
func compareNode(t *testing.T, e astNode, n *Node) {
t.Helper()
require.Equal(t, e.Kind, n.Kind)
require.Equal(t, e.Data, n.Data)
assert.Equal(t, e.Kind, n.Kind)
assert.Equal(t, e.Data, n.Data)
compareIterator(t, e.Children, n.Children())
}
@@ -341,9 +341,9 @@ func TestParser_AST(t *testing.T) {
p.NextExpression()
err := p.Error()
if e.err {
require.Error(t, err)
assert.Error(t, err)
} else {
require.NoError(t, err)
assert.NoError(t, err)
compareNode(t, e.ast, p.Expression())
}
})
@@ -431,9 +431,9 @@ func TestParser_AST_DateTimes(t *testing.T) {
p.NextExpression()
err := p.Error()
if e.err {
require.Error(t, err)
assert.Error(t, err)
} else {
require.NoError(t, err)
assert.NoError(t, err)
expected := astNode{
Kind: KeyValue,
@@ -448,6 +448,163 @@ func TestParser_AST_DateTimes(t *testing.T) {
}
}
// This example demonstrates how to parse a TOML document and preserving
// comments. Comments are stored in the AST as Comment nodes. This example
// displays the structure of the full AST generated by the parser using the
// following structure:
//
// 1. Each root-level expression is separated by three dashes.
// 2. Bytes associated to a node are displayed in square brackets.
// 3. Siblings have the same indentation.
// 4. Children of a node are indented one level.
func ExampleParser_comments() {
doc := `# Top of the document comment.
# Optional, any amount of lines.
# Above table.
[table] # Next to table.
# Above simple value.
key = "value" # Next to simple value.
# Below simple value.
# Some comment alone.
# Multiple comments, on multiple lines.
# Above inline table.
name = { first = "Tom", last = "Preston-Werner" } # Next to inline table.
# Below inline table.
# Above array.
array = [ 1, 2, 3 ] # Next to one-line array.
# Below array.
# Above multi-line array.
key5 = [ # Next to start of inline array.
# Second line before array content.
1, # Next to first element.
# After first element.
# Before second element.
2,
3, # Next to last element
# After last element.
] # Next to end of array.
# Below multi-line array.
# Before array table.
[[products]] # Next to array table.
# After array table.
`
var printGeneric func(*Parser, int, *Node)
printGeneric = func(p *Parser, indent int, e *Node) {
if e == nil {
return
}
s := p.Shape(e.Raw)
x := fmt.Sprintf("%d:%d->%d:%d (%d->%d)", s.Start.Line, s.Start.Column, s.End.Line, s.End.Column, s.Start.Offset, s.End.Offset)
fmt.Printf("%-25s | %s%s [%s]\n", x, strings.Repeat(" ", indent), e.Kind, e.Data)
printGeneric(p, indent+1, e.Child())
printGeneric(p, indent, e.Next())
}
printTree := func(p *Parser) {
for p.NextExpression() {
e := p.Expression()
fmt.Println("---")
printGeneric(p, 0, e)
}
if err := p.Error(); err != nil {
panic(err)
}
}
p := &Parser{
KeepComments: true,
}
p.Reset([]byte(doc))
printTree(p)
// Output:
// ---
// 1:1->1:31 (0->30) | Comment [# Top of the document comment.]
// ---
// 2:1->2:33 (31->63) | Comment [# Optional, any amount of lines.]
// ---
// 4:1->4:15 (65->79) | Comment [# Above table.]
// ---
// 1:1->1:1 (0->0) | Table []
// 5:2->5:7 (81->86) | Key [table]
// 5:9->5:25 (88->104) | Comment [# Next to table.]
// ---
// 6:1->6:22 (105->126) | Comment [# Above simple value.]
// ---
// 1:1->1:1 (0->0) | KeyValue []
// 7:7->7:14 (133->140) | String [value]
// 7:1->7:4 (127->130) | Key [key]
// 7:15->7:38 (141->164) | Comment [# Next to simple value.]
// ---
// 8:1->8:22 (165->186) | Comment [# Below simple value.]
// ---
// 10:1->10:22 (188->209) | Comment [# Some comment alone.]
// ---
// 12:1->12:40 (211->250) | Comment [# Multiple comments, on multiple lines.]
// ---
// 14:1->14:22 (252->273) | Comment [# Above inline table.]
// ---
// 1:1->1:1 (0->0) | KeyValue []
// 15:8->15:9 (281->282) | InlineTable []
// 1:1->1:1 (0->0) | KeyValue []
// 15:18->15:23 (291->296) | String [Tom]
// 15:10->15:15 (283->288) | Key [first]
// 1:1->1:1 (0->0) | KeyValue []
// 15:32->15:48 (305->321) | String [Preston-Werner]
// 15:25->15:29 (298->302) | Key [last]
// 15:1->15:5 (274->278) | Key [name]
// 15:51->15:74 (324->347) | Comment [# Next to inline table.]
// ---
// 16:1->16:22 (348->369) | Comment [# Below inline table.]
// ---
// 18:1->18:15 (371->385) | Comment [# Above array.]
// ---
// 1:1->1:1 (0->0) | KeyValue []
// 1:1->1:1 (0->0) | Array []
// 19:11->19:12 (396->397) | Integer [1]
// 19:14->19:15 (399->400) | Integer [2]
// 19:17->19:18 (402->403) | Integer [3]
// 19:1->19:6 (386->391) | Key [array]
// 19:21->19:46 (406->431) | Comment [# Next to one-line array.]
// ---
// 20:1->20:15 (432->446) | Comment [# Below array.]
// ---
// 22:1->22:26 (448->473) | Comment [# Above multi-line array.]
// ---
// 1:1->1:1 (0->0) | KeyValue []
// 1:1->1:1 (0->0) | Array []
// 23:10->23:42 (483->515) | Comment [# Next to start of inline array.]
// 24:3->24:38 (518->553) | Comment [# Second line before array content.]
// 25:3->25:4 (556->557) | Integer [1]
// 25:6->25:30 (559->583) | Comment [# Next to first element.]
// 26:3->26:25 (586->608) | Comment [# After first element.]
// 27:3->27:27 (611->635) | Comment [# Before second element.]
// 28:3->28:4 (638->639) | Integer [2]
// 29:3->29:4 (643->644) | Integer [3]
// 29:6->29:28 (646->668) | Comment [# Next to last element]
// 30:3->30:24 (671->692) | Comment [# After last element.]
// 23:1->23:5 (474->478) | Key [key5]
// 31:3->31:26 (695->718) | Comment [# Next to end of array.]
// ---
// 32:1->32:26 (719->744) | Comment [# Below multi-line array.]
// ---
// 34:1->34:22 (746->767) | Comment [# Before array table.]
// ---
// 1:1->1:1 (0->0) | ArrayTable []
// 35:3->35:11 (770->778) | Key [products]
// 35:14->35:36 (781->803) | Comment [# Next to array table.]
// ---
// 36:1->36:21 (804->824) | Comment [# After array table.]
}
func ExampleParser() {
doc := `
hello = "world"
-1
View File
@@ -151,7 +151,6 @@ func scanWhitespace(b []byte) ([]byte, []byte) {
return b, b[len(b):]
}
//nolint:unparam
func scanComment(b []byte) ([]byte, []byte, error) {
// comment-start-symbol = %x23 ; #
// non-ascii = %x80-D7FF / %xE000-10FFFF
+7
View File
@@ -0,0 +1,7 @@
package unstable
// The Unmarshaler interface may be implemented by types to customize their
// behavior when being unmarshaled from a TOML document.
type Unmarshaler interface {
UnmarshalTOML(value *Node) error
}