Compare commits

..

2 Commits

Author SHA1 Message Date
Cursor Agent 530b363c2f Remove all usages of unsafe
Removed all usages of `unsafe` and the `internal/danger` package from the codebase.

1.  **`unstable/ast.go`**: Refactored `Node` struct to use `*Node` pointers for `next` and `child` fields instead of integer offsets. This eliminates the need for `unsafe` pointer arithmetic in `Next()` and `Child()` methods.
2.  **`unstable/builder.go`**: Updated `builder` to manage pointers to nodes directly instead of integer offsets.
3.  **`unstable/parser.go`**:
    *   Replaced `danger.SubsliceOffset` with safe capacity-based calculation (`cap(p.data) - cap(b)`), which works because tokens are slices of the parser's input buffer.
4.  **`strict.go`** & **`errors.go`**: Replaced `danger.BytesRange` and `danger.SubsliceOffset` with safe slice capacity arithmetic.
5.  **`unmarshaler.go`**: Replaced `map[danger.TypeID]...` with `map[uintptr]...` for the field paths cache using `reflect.ValueOf(t).Pointer()`. This removes the need for `unsafe` access to `reflect.Type` internals.
6.  **`internal/tracker/seen_test.go`**: Replaced `unsafe.Sizeof` with `reflect.TypeOf(...).Size()`.
7.  **`internal/danger`**: Deleted the package entirely.

Benchmarks show a mix of performance changes:
- Small document unmarshaling (SimpleDocument/struct-4) got slower (+25%), likely due to pointer chasing vs contiguous array access.
- Large document unmarshaling (canada, citm, twitter) actually improved significantly (-24% to -45% latency), likely due to reduced allocation overhead or better cache locality in some paths.
- Memory usage for large datasets decreased significantly (-50% to -60% B/op).
- Overall geomean latency improved by ~6%.

No public interfaces were changed. All tests pass.
2026-01-04 13:24:24 +00:00
Cursor Agent f09f77ab06 Refactor unsafe pointer usage to use reflect.Type and pointers
Remove internal/danger package and replace unsafe pointer arithmetic with direct pointer manipulation. Update AST node references to use pointers instead of integer offsets. This improves code safety and maintainability.

Co-authored-by: thomas.pelletier <thomas.pelletier@bedrockrobotics.com>
2026-01-04 03:11:48 +00:00
68 changed files with 957 additions and 2093 deletions
+1 -1
View File
@@ -5,7 +5,7 @@ Thank you for your pull request!
Please read the Code changes section of the CONTRIBUTING.md file, Please read the Code changes section of the CONTRIBUTING.md file,
and make sure you have followed the instructions. and make sure you have followed the instructions.
https://git.ostiwe.com/ostiwe/go-toml/blob/v2/CONTRIBUTING.md#code-changes https://github.com/pelletier/go-toml/blob/v2/CONTRIBUTING.md#code-changes
--> -->
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
dry-run: false dry-run: false
language: go language: go
- name: Upload Crash - name: Upload Crash
uses: actions/upload-artifact@v7 uses: actions/upload-artifact@v6
if: failure() && steps.build.outcome == 'success' if: failure() && steps.build.outcome == 'success'
with: with:
name: artifacts name: artifacts
+1 -1
View File
@@ -15,6 +15,6 @@ jobs:
- name: Setup go - name: Setup go
uses: actions/setup-go@v6 uses: actions/setup-go@v6
with: with:
go-version: "1.26" go-version: "1.24"
- name: Run tests with coverage - name: Run tests with coverage
run: ./ci.sh coverage -d "${GITHUB_BASE_REF-HEAD}" run: ./ci.sh coverage -d "${GITHUB_BASE_REF-HEAD}"
+2 -2
View File
@@ -20,7 +20,7 @@ jobs:
fetch-depth: 0 fetch-depth: 0
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4 uses: docker/setup-buildx-action@v3
- name: Run Go versions compatibility test - name: Run Go versions compatibility test
run: | run: |
@@ -28,7 +28,7 @@ jobs:
./test-go-versions.sh --output ./test-results $VERSIONS ./test-go-versions.sh --output ./test-results $VERSIONS
- name: Upload test results - name: Upload test results
uses: actions/upload-artifact@v7 uses: actions/upload-artifact@v6
with: with:
name: go-versions-test-results name: go-versions-test-results
path: | path: |
-22
View File
@@ -1,22 +0,0 @@
name: lint
on:
pull_request:
branches:
- v2
jobs:
golangci:
name: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Setup go
uses: actions/setup-go@v6
with:
go-version: "1.26"
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v9
with:
version: v2.8.0
+3 -3
View File
@@ -22,15 +22,15 @@ jobs:
- name: Set up Go - name: Set up Go
uses: actions/setup-go@v6 uses: actions/setup-go@v6
with: with:
go-version: "1.26" go-version: "1.24"
- name: Login to GitHub Container Registry - name: Login to GitHub Container Registry
uses: docker/login-action@v4 uses: docker/login-action@v3
with: with:
registry: ghcr.io registry: ghcr.io
username: ${{ github.actor }} username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }} password: ${{ secrets.GITHUB_TOKEN }}
- name: Run GoReleaser - name: Run GoReleaser
uses: goreleaser/goreleaser-action@v7 uses: goreleaser/goreleaser-action@v6
with: with:
distribution: goreleaser distribution: goreleaser
version: '~> v2' version: '~> v2'
+1 -2
View File
@@ -10,10 +10,9 @@ on:
jobs: jobs:
build: build:
strategy: strategy:
fail-fast: false
matrix: matrix:
os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest', 'macos-14' ] os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest', 'macos-14' ]
go: [ '1.25', '1.26' ] go: [ '1.23', '1.24' ]
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
name: ${{ matrix.go }}/${{ matrix.os }} name: ${{ matrix.go }}/${{ matrix.os }}
steps: steps:
+41 -33
View File
@@ -1,76 +1,84 @@
version = "2" [service]
golangci-lint-version = "1.39.0"
[linters-settings.wsl]
allow-assign-and-anything = true
[linters-settings.exhaustive]
default-signifies-exhaustive = true
[linters] [linters]
default = "none" disable-all = true
enable = [ enable = [
"asciicheck", "asciicheck",
"bodyclose", "bodyclose",
"cyclop",
"deadcode",
"depguard",
"dogsled", "dogsled",
"dupl", "dupl",
"durationcheck", "durationcheck",
"errcheck", "errcheck",
"errorlint", "errorlint",
"exhaustive", "exhaustive",
# "exhaustivestruct",
"exportloopref",
"forbidigo", "forbidigo",
# "forcetypeassert",
"funlen",
"gci",
# "gochecknoglobals",
"gochecknoinits", "gochecknoinits",
"gocognit",
"goconst", "goconst",
"gocritic", "gocritic",
"godoclint", "gocyclo",
"godot",
"godox",
# "goerr113",
"gofmt",
"gofumpt",
"goheader", "goheader",
"goimports",
"golint",
"gomnd",
# "gomoddirectives",
"gomodguard", "gomodguard",
"goprintffuncname", "goprintffuncname",
"gosec", "gosec",
"gosimple",
"govet", "govet",
# "ifshort",
"importas", "importas",
"ineffassign", "ineffassign",
"lll", "lll",
"makezero", "makezero",
"mirror",
"misspell", "misspell",
"nakedret", "nakedret",
"nestif",
"nilerr", "nilerr",
# "nlreturn",
"noctx", "noctx",
"nolintlint", "nolintlint",
"perfsprint", #"paralleltest",
"prealloc", "prealloc",
"predeclared", "predeclared",
"revive", "revive",
"rowserrcheck", "rowserrcheck",
"sqlclosecheck", "sqlclosecheck",
"staticcheck", "staticcheck",
"structcheck",
"stylecheck",
# "testpackage",
"thelper", "thelper",
"tparallel", "tparallel",
"typecheck",
"unconvert", "unconvert",
"unparam", "unparam",
"unused", "unused",
"usetesting", "varcheck",
"wastedassign", "wastedassign",
"whitespace", "whitespace",
] # "wrapcheck",
# "wsl"
[linters.settings.exhaustive]
default-signifies-exhaustive = true
[linters.settings.lll]
line-length = 150
[[linters.exclusions.rules]]
path = ".test.go"
linters = ["goconst", "gosec"]
[[linters.exclusions.rules]]
path = "main.go"
linters = ["forbidigo"]
[[linters.exclusions.rules]]
path = "internal"
linters = ["revive"]
text = "(exported|indent-error-flow): "
[formatters]
enable = [
"gci",
"gofmt",
"gofumpt",
"goimports",
] ]
+3
View File
@@ -22,6 +22,7 @@ builds:
- linux_riscv64 - linux_riscv64
- windows_amd64 - windows_amd64
- windows_arm64 - windows_arm64
- windows_arm
- darwin_amd64 - darwin_amd64
- darwin_arm64 - darwin_arm64
- id: tomljson - id: tomljson
@@ -41,6 +42,7 @@ builds:
- linux_riscv64 - linux_riscv64
- windows_amd64 - windows_amd64
- windows_arm64 - windows_arm64
- windows_arm
- darwin_amd64 - darwin_amd64
- darwin_arm64 - darwin_arm64
- id: jsontoml - id: jsontoml
@@ -60,6 +62,7 @@ builds:
- linux_arm - linux_arm
- windows_amd64 - windows_amd64
- windows_arm64 - windows_arm64
- windows_arm
- darwin_amd64 - darwin_amd64
- darwin_arm64 - darwin_arm64
universal_binaries: universal_binaries:
-7
View File
@@ -40,13 +40,6 @@ go-toml is a TOML library for Go. The goal is to provide an easy-to-use and effi
- Follow existing code format and structure - Follow existing code format and structure
- Code must pass `go fmt` - Code must pass `go fmt`
- Code must pass linting with the same golangci-lint version as CI (see version in `.github/workflows/lint.yml`):
```bash
# Install specific version (check lint.yml for current version)
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/HEAD/install.sh | sh -s -- -b $(go env GOPATH)/bin <version>
# Run linter
golangci-lint run ./...
```
### Commit Messages ### Commit Messages
+6 -6
View File
@@ -21,7 +21,7 @@ improvement, or new features that weren't envisioned before. Sometimes, a
seemingly innocent question leads to the fix of a bug. Don't hesitate and ask seemingly innocent question leads to the fix of a bug. Don't hesitate and ask
away! away!
[discussions]: https://git.ostiwe.com/ostiwe/go-toml/discussions [discussions]: https://github.com/pelletier/go-toml/discussions
## Improve the documentation ## Improve the documentation
@@ -224,12 +224,12 @@ Checklist:
5. If new version is an alpha or beta only, check pre-release box. 5. If new version is an alpha or beta only, check pre-release box.
[issues-tracker]: https://git.ostiwe.com/ostiwe/go-toml/issues [issues-tracker]: https://github.com/pelletier/go-toml/issues
[bug-report]: https://git.ostiwe.com/ostiwe/go-toml/issues/new?template=bug_report.md [bug-report]: https://github.com/pelletier/go-toml/issues/new?template=bug_report.md
[pkg.go.dev]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml [pkg.go.dev]: https://pkg.go.dev/github.com/pelletier/go-toml
[readme]: ./README.md [readme]: ./README.md
[fork]: https://help.github.com/articles/fork-a-repo [fork]: https://help.github.com/articles/fork-a-repo
[pull-request]: https://help.github.com/en/articles/creating-a-pull-request [pull-request]: https://help.github.com/en/articles/creating-a-pull-request
[new-release]: https://git.ostiwe.com/ostiwe/go-toml/releases/new [new-release]: https://github.com/pelletier/go-toml/releases/new
[gh]: https://github.com/cli/cli [gh]: https://github.com/cli/cli
[pr-labels]: https://git.ostiwe.com/ostiwe/go-toml/blob/v2/.github/release.yml [pr-labels]: https://github.com/pelletier/go-toml/blob/v2/.github/release.yml
+39 -39
View File
@@ -4,21 +4,21 @@ Go library for the [TOML](https://toml.io/en/) format.
This library supports [TOML v1.0.0](https://toml.io/en/v1.0.0). This library supports [TOML v1.0.0](https://toml.io/en/v1.0.0).
[🐞 Bug Reports](https://git.ostiwe.com/ostiwe/go-toml/issues) [🐞 Bug Reports](https://github.com/pelletier/go-toml/issues)
[💬 Anything else](https://git.ostiwe.com/ostiwe/go-toml/discussions) [💬 Anything else](https://github.com/pelletier/go-toml/discussions)
## Documentation ## Documentation
Full API, examples, and implementation notes are available in the Go Full API, examples, and implementation notes are available in the Go
documentation. documentation.
[![Go Reference](https://pkg.go.dev/badge/git.ostiwe.com/ostiwe/go-toml/v2.svg)](https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2) [![Go Reference](https://pkg.go.dev/badge/github.com/pelletier/go-toml/v2.svg)](https://pkg.go.dev/github.com/pelletier/go-toml/v2)
## Import ## Import
```go ```go
import "git.ostiwe.com/ostiwe/go-toml/v2" import "github.com/pelletier/go-toml/v2"
``` ```
See [Modules](#Modules). See [Modules](#Modules).
@@ -41,7 +41,7 @@ operations should not be shockingly slow. See [benchmarks](#benchmarks).
the TOML document was not present in the target structure. This is a great way the TOML document was not present in the target structure. This is a great way
to check for typos. [See example in the documentation][strict]. to check for typos. [See example in the documentation][strict].
[strict]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#example-Decoder.DisallowUnknownFields [strict]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#example-Decoder.DisallowUnknownFields
### Contextualized errors ### Contextualized errors
@@ -56,7 +56,7 @@ example:
3| port = 50 3| port = 50
``` ```
[decode-err]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#DecodeError [decode-err]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#DecodeError
### Local date and time support ### Local date and time support
@@ -68,9 +68,9 @@ making them convenient yet unambiguous structures for their respective TOML
representation. representation.
[ldt]: https://toml.io/en/v1.0.0#local-date-time [ldt]: https://toml.io/en/v1.0.0#local-date-time
[tld]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#LocalDate [tld]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#LocalDate
[tlt]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#LocalTime [tlt]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#LocalTime
[tldt]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#LocalDateTime [tldt]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#LocalDateTime
### Commented config ### Commented config
@@ -90,7 +90,7 @@ port = 4242
# version = 'TLS 1.3' # version = 'TLS 1.3'
``` ```
[comments-example]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#example-Marshal-Commented [comments-example]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#example-Marshal-Commented
## Getting started ## Getting started
@@ -135,7 +135,7 @@ fmt.Println("tags:", cfg.Tags)
// tags: [go toml] // tags: [go toml]
``` ```
[unmarshal]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#Unmarshal [unmarshal]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#Unmarshal
Here is an example using tables with some simple nesting: Here is an example using tables with some simple nesting:
@@ -217,7 +217,7 @@ fmt.Println(string(b))
// Tags = ['go', 'toml'] // Tags = ['go', 'toml']
``` ```
[marshal]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#Marshal [marshal]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#Marshal
## Unstable API ## Unstable API
@@ -228,7 +228,7 @@ API subject to change.
### Parser ### Parser
Parser is the unstable API that allows iterative parsing of a TOML document at Parser is the unstable API that allows iterative parsing of a TOML document at
the AST level. See https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2/unstable. the AST level. See https://pkg.go.dev/github.com/pelletier/go-toml/v2/unstable.
## Benchmarks ## Benchmarks
@@ -239,12 +239,12 @@ Execution time speedup compared to other Go TOML libraries:
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr> <tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead> </thead>
<tbody> <tbody>
<tr><td>Marshal/HugoFrontMatter-2</td><td>2.1x</td><td>2.0x</td></tr> <tr><td>Marshal/HugoFrontMatter-2</td><td>1.9x</td><td>2.2x</td></tr>
<tr><td>Marshal/ReferenceFile/map-2</td><td>2.0x</td><td>2.0x</td></tr> <tr><td>Marshal/ReferenceFile/map-2</td><td>1.7x</td><td>2.1x</td></tr>
<tr><td>Marshal/ReferenceFile/struct-2</td><td>2.3x</td><td>2.5x</td></tr> <tr><td>Marshal/ReferenceFile/struct-2</td><td>2.2x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/HugoFrontMatter-2</td><td>3.3x</td><td>2.8x</td></tr> <tr><td>Unmarshal/HugoFrontMatter-2</td><td>2.9x</td><td>2.7x</td></tr>
<tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.9x</td><td>3.0x</td></tr> <tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.6x</td><td>2.7x</td></tr>
<tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.8x</td><td>5.0x</td></tr> <tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.6x</td><td>5.1x</td></tr>
</tbody> </tbody>
</table> </table>
<details><summary>See more</summary> <details><summary>See more</summary>
@@ -257,17 +257,17 @@ provided for completeness.</p>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr> <tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead> </thead>
<tbody> <tbody>
<tr><td>Marshal/SimpleDocument/map-2</td><td>2.0x</td><td>2.9x</td></tr> <tr><td>Marshal/SimpleDocument/map-2</td><td>1.8x</td><td>2.7x</td></tr>
<tr><td>Marshal/SimpleDocument/struct-2</td><td>2.5x</td><td>3.6x</td></tr> <tr><td>Marshal/SimpleDocument/struct-2</td><td>2.7x</td><td>3.8x</td></tr>
<tr><td>Unmarshal/SimpleDocument/map-2</td><td>4.2x</td><td>3.4x</td></tr> <tr><td>Unmarshal/SimpleDocument/map-2</td><td>3.8x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/SimpleDocument/struct-2</td><td>5.9x</td><td>4.4x</td></tr> <tr><td>Unmarshal/SimpleDocument/struct-2</td><td>5.6x</td><td>4.1x</td></tr>
<tr><td>UnmarshalDataset/example-2</td><td>3.2x</td><td>2.9x</td></tr> <tr><td>UnmarshalDataset/example-2</td><td>3.0x</td><td>3.2x</td></tr>
<tr><td>UnmarshalDataset/code-2</td><td>2.4x</td><td>2.8x</td></tr> <tr><td>UnmarshalDataset/code-2</td><td>2.3x</td><td>2.9x</td></tr>
<tr><td>UnmarshalDataset/twitter-2</td><td>2.7x</td><td>2.5x</td></tr> <tr><td>UnmarshalDataset/twitter-2</td><td>2.6x</td><td>2.7x</td></tr>
<tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.3x</td><td>2.3x</td></tr> <tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.2x</td><td>2.3x</td></tr>
<tr><td>UnmarshalDataset/canada-2</td><td>1.9x</td><td>1.5x</td></tr> <tr><td>UnmarshalDataset/canada-2</td><td>1.8x</td><td>1.5x</td></tr>
<tr><td>UnmarshalDataset/config-2</td><td>5.4x</td><td>3.0x</td></tr> <tr><td>UnmarshalDataset/config-2</td><td>4.1x</td><td>2.9x</td></tr>
<tr><td>geomean</td><td>2.9x</td><td>2.8x</td></tr> <tr><td>geomean</td><td>2.7x</td><td>2.8x</td></tr>
</tbody> </tbody>
</table> </table>
<p>This table can be generated with <code>./ci.sh benchmark -a -html</code>.</p> <p>This table can be generated with <code>./ci.sh benchmark -a -html</code>.</p>
@@ -281,7 +281,7 @@ Installation instructions:
- Go ≥ 1.16: Nothing to do. Use the import in your code. The `go` command deals - Go ≥ 1.16: Nothing to do. Use the import in your code. The `go` command deals
with it automatically. with it automatically.
- Go ≥ 1.13: `GO111MODULE=on go get git.ostiwe.com/ostiwe/go-toml/v2`. - Go ≥ 1.13: `GO111MODULE=on go get github.com/pelletier/go-toml/v2`.
In case of trouble: [Go Modules FAQ][mod-faq]. In case of trouble: [Go Modules FAQ][mod-faq].
@@ -294,21 +294,21 @@ Go-toml provides three handy command line tools:
* `tomljson`: Reads a TOML file and outputs its JSON representation. * `tomljson`: Reads a TOML file and outputs its JSON representation.
``` ```
$ go install git.ostiwe.com/ostiwe/go-toml/v2/cmd/tomljson@latest $ go install github.com/pelletier/go-toml/v2/cmd/tomljson@latest
$ tomljson --help $ tomljson --help
``` ```
* `jsontoml`: Reads a JSON file and outputs a TOML representation. * `jsontoml`: Reads a JSON file and outputs a TOML representation.
``` ```
$ go install git.ostiwe.com/ostiwe/go-toml/v2/cmd/jsontoml@latest $ go install github.com/pelletier/go-toml/v2/cmd/jsontoml@latest
$ jsontoml --help $ jsontoml --help
``` ```
* `tomll`: Lints and reformats a TOML file. * `tomll`: Lints and reformats a TOML file.
``` ```
$ go install git.ostiwe.com/ostiwe/go-toml/v2/cmd/tomll@latest $ go install github.com/pelletier/go-toml/v2/cmd/tomll@latest
$ tomll --help $ tomll --help
``` ```
@@ -323,7 +323,7 @@ docker run -i ghcr.io/pelletier/go-toml:v2 tomljson < example.toml
Multiple versions are available on [ghcr.io][docker]. Multiple versions are available on [ghcr.io][docker].
[docker]: https://git.ostiwe.com/ostiwe/go-toml/pkgs/container/go-toml [docker]: https://github.com/pelletier/go-toml/pkgs/container/go-toml
## Migrating from v1 ## Migrating from v1
@@ -344,7 +344,7 @@ This could impact you if you are relying on casing to differentiate two fields,
and one of them is a not using the `toml` struct tag. The recommended solution and one of them is a not using the `toml` struct tag. The recommended solution
is to be specific about tag names for those fields using the `toml` struct tag. is to be specific about tag names for those fields using the `toml` struct tag.
[v1-keys]: https://git.ostiwe.com/ostiwe/go-toml/blob/a2e52561804c6cd9392ebf0048ca64fe4af67a43/marshal.go#L775-L781 [v1-keys]: https://github.com/pelletier/go-toml/blob/a2e52561804c6cd9392ebf0048ca64fe4af67a43/marshal.go#L775-L781
#### Ignore preexisting value in interface #### Ignore preexisting value in interface
@@ -544,7 +544,7 @@ fmt.Println("v2 Encoder:\n" + string(buf.Bytes()))
// key = 'value' // key = 'value'
``` ```
[sit]: https://pkg.go.dev/git.ostiwe.com/ostiwe/go-toml/v2#Encoder.SetIndentTables [sit]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#Encoder.SetIndentTables
#### Keys and strings are single quoted #### Keys and strings are single quoted
@@ -608,7 +608,7 @@ added to make the encoder behave correctly. Given backward compatibility is not
a problem anymore, v2 does the right thing by default: it follows the behavior a problem anymore, v2 does the right thing by default: it follows the behavior
of `encoding/json`. `Encoder.PromoteAnonymous` has been removed. of `encoding/json`. `Encoder.PromoteAnonymous` has been removed.
[nodoc]: https://git.ostiwe.com/ostiwe/go-toml/discussions/506#discussioncomment-1526038 [nodoc]: https://github.com/pelletier/go-toml/discussions/506#discussioncomment-1526038
### `query` ### `query`
@@ -620,7 +620,7 @@ This package has been removed because it was essentially not supported anymore
(last commit May 2020), increased the complexity of the code base, and more (last commit May 2020), increased the complexity of the code base, and more
complete solutions exist out there. complete solutions exist out there.
[query]: https://git.ostiwe.com/ostiwe/go-toml/tree/f99d6bbca119636aeafcf351ee52b3d202782627/query [query]: https://github.com/pelletier/go-toml/tree/f99d6bbca119636aeafcf351ee52b3d202782627/query
[dasel]: https://github.com/TomWright/dasel [dasel]: https://github.com/TomWright/dasel
## Versioning ## Versioning
+6 -6
View File
@@ -8,11 +8,11 @@ import (
"path/filepath" "path/filepath"
"testing" "testing"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
var benchInputs = []struct { var bench_inputs = []struct {
name string name string
jsonLen int jsonLen int
}{ }{
@@ -30,7 +30,7 @@ var benchInputs = []struct {
} }
func TestUnmarshalDatasetCode(t *testing.T) { func TestUnmarshalDatasetCode(t *testing.T) {
for _, tc := range benchInputs { for _, tc := range bench_inputs {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
buf := fixture(t, tc.name) buf := fixture(t, tc.name)
@@ -45,7 +45,7 @@ func TestUnmarshalDatasetCode(t *testing.T) {
} }
func BenchmarkUnmarshalDataset(b *testing.B) { func BenchmarkUnmarshalDataset(b *testing.B) {
for _, tc := range benchInputs { for _, tc := range bench_inputs {
b.Run(tc.name, func(b *testing.B) { b.Run(tc.name, func(b *testing.B) {
buf := fixture(b, tc.name) buf := fixture(b, tc.name)
b.SetBytes(int64(len(buf))) b.SetBytes(int64(len(buf)))
@@ -69,7 +69,7 @@ func fixture(tb testing.TB, path string) []byte {
tb.Skip("benchmark fixture not found:", file) tb.Skip("benchmark fixture not found:", file)
} }
assert.NoError(tb, err) assert.NoError(tb, err)
defer func() { _ = f.Close() }() defer f.Close()
gz, err := gzip.NewReader(f) gz, err := gzip.NewReader(f)
assert.NoError(tb, err) assert.NoError(tb, err)
+18 -18
View File
@@ -6,8 +6,8 @@ import (
"testing" "testing"
"time" "time"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
func TestUnmarshalSimple(t *testing.T) { func TestUnmarshalSimple(t *testing.T) {
@@ -18,7 +18,7 @@ func TestUnmarshalSimple(t *testing.T) {
err := toml.Unmarshal(doc, &d) err := toml.Unmarshal(doc, &d)
if err != nil { if err != nil {
t.Error(err) panic(err)
} }
} }
@@ -38,7 +38,7 @@ func BenchmarkUnmarshal(b *testing.B) {
err := toml.Unmarshal(doc, &d) err := toml.Unmarshal(doc, &d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
} }
}) })
@@ -52,7 +52,7 @@ func BenchmarkUnmarshal(b *testing.B) {
d := map[string]interface{}{} d := map[string]interface{}{}
err := toml.Unmarshal(doc, &d) err := toml.Unmarshal(doc, &d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
} }
}) })
@@ -72,7 +72,7 @@ func BenchmarkUnmarshal(b *testing.B) {
d := benchmarkDoc{} d := benchmarkDoc{}
err := toml.Unmarshal(bytes, &d) err := toml.Unmarshal(bytes, &d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
} }
}) })
@@ -85,7 +85,7 @@ func BenchmarkUnmarshal(b *testing.B) {
d := map[string]interface{}{} d := map[string]interface{}{}
err := toml.Unmarshal(bytes, &d) err := toml.Unmarshal(bytes, &d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
} }
}) })
@@ -99,7 +99,7 @@ func BenchmarkUnmarshal(b *testing.B) {
d := map[string]interface{}{} d := map[string]interface{}{}
err := toml.Unmarshal(hugoFrontMatterbytes, &d) err := toml.Unmarshal(hugoFrontMatterbytes, &d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
} }
}) })
@@ -123,7 +123,7 @@ func BenchmarkMarshal(b *testing.B) {
err := toml.Unmarshal(doc, &d) err := toml.Unmarshal(doc, &d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
b.ReportAllocs() b.ReportAllocs()
@@ -134,7 +134,7 @@ func BenchmarkMarshal(b *testing.B) {
for i := 0; i < b.N; i++ { for i := 0; i < b.N; i++ {
out, err = marshal(d) out, err = marshal(d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
} }
@@ -145,7 +145,7 @@ func BenchmarkMarshal(b *testing.B) {
d := map[string]interface{}{} d := map[string]interface{}{}
err := toml.Unmarshal(doc, &d) err := toml.Unmarshal(doc, &d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
b.ReportAllocs() b.ReportAllocs()
@@ -156,7 +156,7 @@ func BenchmarkMarshal(b *testing.B) {
for i := 0; i < b.N; i++ { for i := 0; i < b.N; i++ {
out, err = marshal(d) out, err = marshal(d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
} }
@@ -174,7 +174,7 @@ func BenchmarkMarshal(b *testing.B) {
d := benchmarkDoc{} d := benchmarkDoc{}
err := toml.Unmarshal(bytes, &d) err := toml.Unmarshal(bytes, &d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
b.ReportAllocs() b.ReportAllocs()
b.ResetTimer() b.ResetTimer()
@@ -184,7 +184,7 @@ func BenchmarkMarshal(b *testing.B) {
for i := 0; i < b.N; i++ { for i := 0; i < b.N; i++ {
out, err = marshal(d) out, err = marshal(d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
} }
@@ -195,7 +195,7 @@ func BenchmarkMarshal(b *testing.B) {
d := map[string]interface{}{} d := map[string]interface{}{}
err := toml.Unmarshal(bytes, &d) err := toml.Unmarshal(bytes, &d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
b.ReportAllocs() b.ReportAllocs()
@@ -205,7 +205,7 @@ func BenchmarkMarshal(b *testing.B) {
for i := 0; i < b.N; i++ { for i := 0; i < b.N; i++ {
out, err = marshal(d) out, err = marshal(d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
} }
@@ -217,7 +217,7 @@ func BenchmarkMarshal(b *testing.B) {
d := map[string]interface{}{} d := map[string]interface{}{}
err := toml.Unmarshal(hugoFrontMatterbytes, &d) err := toml.Unmarshal(hugoFrontMatterbytes, &d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
b.ReportAllocs() b.ReportAllocs()
@@ -228,7 +228,7 @@ func BenchmarkMarshal(b *testing.B) {
for i := 0; i < b.N; i++ { for i := 0; i < b.N; i++ {
out, err = marshal(d) out, err = marshal(d)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
} }
+5 -10
View File
@@ -117,8 +117,8 @@ coverage() {
target_diff="${output_dir}/target.diff.txt" target_diff="${output_dir}/target.diff.txt"
head_diff="${output_dir}/head.diff.txt" head_diff="${output_dir}/head.diff.txt"
cat "${target_out}" | grep -E '^git.ostiwe.com/ostiwe/go-toml' | tr -s "\t " | cut -f 2,3 | sort > "${target_diff}" cat "${target_out}" | grep -E '^github.com/pelletier/go-toml' | tr -s "\t " | cut -f 2,3 | sort > "${target_diff}"
cat "${head_out}" | grep -E '^git.ostiwe.com/ostiwe/go-toml' | tr -s "\t " | cut -f 2,3 | sort > "${head_diff}" cat "${head_out}" | grep -E '^github.com/pelletier/go-toml' | tr -s "\t " | cut -f 2,3 | sort > "${head_diff}"
diff --side-by-side --suppress-common-lines "${target_diff}" "${head_diff}" diff --side-by-side --suppress-common-lines "${target_diff}" "${head_diff}"
return 1 return 1
@@ -147,7 +147,7 @@ bench() {
pushd "$dir" pushd "$dir"
if [ "${replace}" != "" ]; then if [ "${replace}" != "" ]; then
find ./benchmark/ -iname '*.go' -exec sed -i -E "s|git.ostiwe.com/ostiwe/go-toml/v2\"|${replace}\"|g" {} \; find ./benchmark/ -iname '*.go' -exec sed -i -E "s|github.com/pelletier/go-toml/v2|${replace}|g" {} \;
go get "${replace}" go get "${replace}"
fi fi
@@ -195,11 +195,6 @@ for line in reversed(lines[2:]):
"%.1fx" % (float(line[3])/v2), # v1 "%.1fx" % (float(line[3])/v2), # v1
"%.1fx" % (float(line[7])/v2), # bs "%.1fx" % (float(line[7])/v2), # bs
]) ])
if not results:
print("No benchmark results to display.", file=sys.stderr)
sys.exit(1)
# move geomean to the end # move geomean to the end
results.append(results[0]) results.append(results[0])
del results[0] del results[0]
@@ -257,9 +252,9 @@ benchmark() {
shift shift
v2stats=`fmktemp go-toml-v2` v2stats=`fmktemp go-toml-v2`
bench HEAD "${v2stats}" "git.ostiwe.com/ostiwe/go-toml/v2" bench HEAD "${v2stats}" "github.com/pelletier/go-toml/v2"
v1stats=`fmktemp go-toml-v1` v1stats=`fmktemp go-toml-v1`
bench HEAD "${v1stats}" "git.ostiwe.com/ostiwe/go-toml" bench HEAD "${v1stats}" "github.com/pelletier/go-toml"
bsstats=`fmktemp bs-toml` bsstats=`fmktemp bs-toml`
bench HEAD "${bsstats}" "github.com/BurntSushi/toml" bench HEAD "${bsstats}" "github.com/BurntSushi/toml"
+1 -2
View File
@@ -1,4 +1,3 @@
// Package gotoml-test-decoder is a minimal decoder program used to compare this library with other TOML implementations.
package main package main
import ( import (
@@ -7,7 +6,7 @@ import (
"os" "os"
"path" "path"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/testsuite" "github.com/pelletier/go-toml/v2/internal/testsuite"
) )
func main() { func main() {
@@ -1,4 +1,3 @@
// Package gotoml-test-encoder is a minimal encoder program used to compare this library with other TOML implementations.
package main package main
import ( import (
@@ -7,7 +6,7 @@ import (
"os" "os"
"path" "path"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/testsuite" "github.com/pelletier/go-toml/v2/internal/testsuite"
) )
func main() { func main() {
@@ -25,7 +24,7 @@ func main() {
} }
func usage() { func usage() {
log.Printf("Usage: %s < json-file\n", path.Base(os.Args[0])) log.Printf("Usage: %s < toml-file\n", path.Base(os.Args[0]))
flag.PrintDefaults() flag.PrintDefaults()
os.Exit(1) os.Exit(1)
} }
+7 -7
View File
@@ -14,7 +14,7 @@
// //
// Using Go: // Using Go:
// //
// go install git.ostiwe.com/ostiwe/go-toml/v2/cmd/jsontoml@latest // go install github.com/pelletier/go-toml/v2/cmd/jsontoml@latest
package main package main
import ( import (
@@ -22,8 +22,8 @@ import (
"flag" "flag"
"io" "io"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/cli" "github.com/pelletier/go-toml/v2/internal/cli"
) )
const usage = `jsontoml can be used in two ways: const usage = `jsontoml can be used in two ways:
@@ -34,10 +34,10 @@ Reading from a file:
jsontoml file.json > file.toml jsontoml file.json > file.toml
` `
var useJSONNumber bool var useJsonNumber bool
func main() { func main() {
flag.BoolVar(&useJSONNumber, "use-json-number", false, "unmarshal numbers into `json.Number` type instead of as `float64`") flag.BoolVar(&useJsonNumber, "use-json-number", false, "unmarshal numbers into `json.Number` type instead of as `float64`")
p := cli.Program{ p := cli.Program{
Usage: usage, Usage: usage,
@@ -52,9 +52,9 @@ func convert(r io.Reader, w io.Writer) error {
d := json.NewDecoder(r) d := json.NewDecoder(r)
e := toml.NewEncoder(w) e := toml.NewEncoder(w)
if useJSONNumber { if useJsonNumber {
d.UseNumber() d.UseNumber()
e.SetMarshalJSONNumbers(true) e.SetMarshalJsonNumbers(true)
} }
err := d.Decode(&v) err := d.Decode(&v)
+4 -4
View File
@@ -5,7 +5,7 @@ import (
"strings" "strings"
"testing" "testing"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
func TestConvert(t *testing.T) { func TestConvert(t *testing.T) {
@@ -14,7 +14,7 @@ func TestConvert(t *testing.T) {
input string input string
expected string expected string
errors bool errors bool
useJSONNumber bool useJsonNumber bool
}{ }{
{ {
name: "valid json", name: "valid json",
@@ -30,7 +30,7 @@ a = 42.0
}, },
{ {
name: "use json number", name: "use json number",
useJSONNumber: true, useJsonNumber: true,
input: ` input: `
{ {
"mytoml": { "mytoml": {
@@ -50,7 +50,7 @@ a = 42
for _, e := range examples { for _, e := range examples {
b := new(bytes.Buffer) b := new(bytes.Buffer)
useJSONNumber = e.useJSONNumber useJsonNumber = e.useJsonNumber
err := convert(strings.NewReader(e.input), b) err := convert(strings.NewReader(e.input), b)
if e.errors { if e.errors {
assert.Error(t, err) assert.Error(t, err)
+3 -3
View File
@@ -14,7 +14,7 @@
// //
// Using Go: // Using Go:
// //
// go install git.ostiwe.com/ostiwe/go-toml/v2/cmd/tomljson@latest // go install github.com/pelletier/go-toml/v2/cmd/tomljson@latest
package main package main
import ( import (
@@ -23,8 +23,8 @@ import (
"fmt" "fmt"
"io" "io"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/cli" "github.com/pelletier/go-toml/v2/internal/cli"
) )
const usage = `tomljson can be used in two ways: const usage = `tomljson can be used in two ways:
+3 -3
View File
@@ -2,12 +2,12 @@ package main
import ( import (
"bytes" "bytes"
"errors" "fmt"
"io" "io"
"strings" "strings"
"testing" "testing"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
func TestConvert(t *testing.T) { func TestConvert(t *testing.T) {
@@ -56,5 +56,5 @@ a = 42`),
type badReader struct{} type badReader struct{}
func (r *badReader) Read([]byte) (int, error) { func (r *badReader) Read([]byte) (int, error) {
return 0, errors.New("reader failed on purpose") return 0, fmt.Errorf("reader failed on purpose")
} }
+3 -3
View File
@@ -14,14 +14,14 @@
// //
// Using Go: // Using Go:
// //
// go install git.ostiwe.com/ostiwe/go-toml/v2/cmd/tomll@latest // go install github.com/pelletier/go-toml/v2/cmd/tomll@latest
package main package main
import ( import (
"io" "io"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/cli" "github.com/pelletier/go-toml/v2/internal/cli"
) )
const usage = `tomll can be used in two ways: const usage = `tomll can be used in two ways:
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"strings" "strings"
"testing" "testing"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
func TestConvert(t *testing.T) { func TestConvert(t *testing.T) {
+19 -22
View File
@@ -3,7 +3,7 @@
// //
// Within the go-toml package, run `go generate`. Otherwise, use: // Within the go-toml package, run `go generate`. Otherwise, use:
// //
// go run git.ostiwe.com/ostiwe/go-toml/cmd/tomltestgen -o toml_testgen_test.go // go run github.com/pelletier/go-toml/cmd/tomltestgen -o toml_testgen_test.go
package main package main
import ( import (
@@ -18,7 +18,6 @@ import (
"strings" "strings"
"text/template" "text/template"
"time" "time"
"unicode"
) )
type invalid struct { type invalid struct {
@@ -29,7 +28,7 @@ type invalid struct {
type valid struct { type valid struct {
Name string Name string
Input string Input string
JSONRef string JsonRef string
} }
type testsCollection struct { type testsCollection struct {
@@ -40,11 +39,12 @@ type testsCollection struct {
Count int Count int
} }
const srcTemplate = "// Code generated by tomltestgen for toml-test ref {{.Ref}} on {{.Timestamp}}. DO NOT EDIT.\n" + const srcTemplate = "// Generated by tomltestgen for toml-test ref {{.Ref}} on {{.Timestamp}}\n" +
"package toml_test\n" + "package toml_test\n" +
" import (\n" + " import (\n" +
" \"testing\"\n" + " \"testing\"\n" +
")\n" + ")\n" +
"{{range .Invalid}}\n" + "{{range .Invalid}}\n" +
"func TestTOMLTest_Invalid_{{.Name}}(t *testing.T) {\n" + "func TestTOMLTest_Invalid_{{.Name}}(t *testing.T) {\n" +
" input := {{.Input|gostr}}\n" + " input := {{.Input|gostr}}\n" +
@@ -55,31 +55,28 @@ const srcTemplate = "// Code generated by tomltestgen for toml-test ref {{.Ref}}
"{{range .Valid}}\n" + "{{range .Valid}}\n" +
"func TestTOMLTest_Valid_{{.Name}}(t *testing.T) {\n" + "func TestTOMLTest_Valid_{{.Name}}(t *testing.T) {\n" +
" input := {{.Input|gostr}}\n" + " input := {{.Input|gostr}}\n" +
" jsonRef := {{.JSONRef|gostr}}\n" + " jsonRef := {{.JsonRef|gostr}}\n" +
" testgenValid(t, input, jsonRef)\n" + " testgenValid(t, input, jsonRef)\n" +
"}\n" + "}\n" +
"{{end}}\n" "{{end}}\n"
func kebabToCamel(kebab string) string { func kebabToCamel(kebab string) string {
var buf strings.Builder camel := ""
nextUpper := true nextUpper := true
for _, c := range kebab { for _, c := range kebab {
if nextUpper { if nextUpper {
buf.WriteRune(unicode.ToUpper(c)) camel += strings.ToUpper(string(c))
nextUpper = false nextUpper = false
} else if c == '-' {
nextUpper = true
} else if c == '/' {
nextUpper = true
camel += "_"
} else { } else {
switch c { camel += string(c)
case '-':
nextUpper = true
case '/':
nextUpper = true
buf.WriteByte('_')
default:
buf.WriteRune(c)
} }
} }
} return camel
return buf.String()
} }
func templateGoStr(input string) string { func templateGoStr(input string) string {
@@ -113,7 +110,7 @@ func main() {
log.Printf("> [%s] %s\n", "invalid", name) log.Printf("> [%s] %s\n", "invalid", name)
tomlContent, err := os.ReadFile(f) // #nosec G304 tomlContent, err := os.ReadFile(f)
if err != nil { if err != nil {
fmt.Printf("failed to read test file: %s\n", err) fmt.Printf("failed to read test file: %s\n", err)
os.Exit(1) os.Exit(1)
@@ -134,14 +131,14 @@ func main() {
log.Printf("> [%s] %s\n", "valid", name) log.Printf("> [%s] %s\n", "valid", name)
tomlContent, err := os.ReadFile(f) // #nosec G304 tomlContent, err := os.ReadFile(f)
if err != nil { if err != nil {
fmt.Printf("failed reading test file: %s\n", err) fmt.Printf("failed reading test file: %s\n", err)
os.Exit(1) os.Exit(1)
} }
filename = strings.TrimSuffix(f, ".toml") filename = strings.TrimSuffix(f, ".toml")
jsonContent, err := os.ReadFile(filename + ".json") // #nosec G304 jsonContent, err := os.ReadFile(filename + ".json")
if err != nil { if err != nil {
fmt.Printf("failed reading validation json: %s\n", err) fmt.Printf("failed reading validation json: %s\n", err)
os.Exit(1) os.Exit(1)
@@ -150,7 +147,7 @@ func main() {
collection.Valid = append(collection.Valid, valid{ collection.Valid = append(collection.Valid, valid{
Name: name, Name: name,
Input: string(tomlContent), Input: string(tomlContent),
JSONRef: string(jsonContent), JsonRef: string(jsonContent),
}) })
collection.Count++ collection.Count++
} }
@@ -176,7 +173,7 @@ func main() {
return return
} }
err = os.WriteFile(*out, outputBytes, 0o600) err = os.WriteFile(*out, outputBytes, 0o644)
if err != nil { if err != nil {
panic(err) panic(err)
} }
+5 -4
View File
@@ -6,7 +6,7 @@ import (
"strconv" "strconv"
"time" "time"
"git.ostiwe.com/ostiwe/go-toml/v2/unstable" "github.com/pelletier/go-toml/v2/unstable"
) )
func parseInteger(b []byte) (int64, error) { func parseInteger(b []byte) (int64, error) {
@@ -230,8 +230,8 @@ func parseLocalTime(b []byte) (LocalTime, []byte, error) {
return t, nil, err return t, nil, err
} }
if t.Second > 59 { if t.Second > 60 {
return t, nil, unstable.NewParserError(b[6:8], "seconds cannot be greater than 59") return t, nil, unstable.NewParserError(b[6:8], "seconds cannot be greater 60")
} }
b = b[8:] b = b[8:]
@@ -258,7 +258,7 @@ func parseLocalTime(b []byte) (LocalTime, []byte, error) {
// to the supported precision and ignores the // to the supported precision and ignores the
// remaining digits. // remaining digits.
// //
// https://git.ostiwe.com/ostiwe/go-toml/discussions/707 // https://github.com/pelletier/go-toml/discussions/707
continue continue
} }
@@ -279,6 +279,7 @@ func parseLocalTime(b []byte) (LocalTime, []byte, error) {
return t, b, nil return t, b, nil
} }
//nolint:cyclop
func parseFloat(b []byte) (float64, error) { func parseFloat(b []byte) (float64, error) {
if len(b) == 4 && (b[0] == '+' || b[0] == '-') && b[1] == 'n' && b[2] == 'a' && b[3] == 'n' { if len(b) == 4 && (b[0] == '+' || b[0] == '-') && b[1] == 'n' && b[2] == 'a' && b[3] == 'n' {
return math.NaN(), nil return math.NaN(), nil
+5 -26
View File
@@ -2,11 +2,10 @@ package toml
import ( import (
"fmt" "fmt"
"reflect"
"strconv" "strconv"
"strings" "strings"
"git.ostiwe.com/ostiwe/go-toml/v2/unstable" "github.com/pelletier/go-toml/v2/unstable"
) )
// DecodeError represents an error encountered during the parsing or decoding // DecodeError represents an error encountered during the parsing or decoding
@@ -58,14 +57,13 @@ func (s *StrictMissingError) String() string {
// //
// Implements errors.Join() interface. // Implements errors.Join() interface.
func (s *StrictMissingError) Unwrap() []error { func (s *StrictMissingError) Unwrap() []error {
errs := make([]error, len(s.Errors)) var errs []error
for i := range s.Errors { for i := range s.Errors {
errs[i] = &s.Errors[i] errs = append(errs, &s.Errors[i])
} }
return errs return errs
} }
// Key represents a TOML key as a sequence of key parts.
type Key []string type Key []string
// Error returns the error message contained in the DecodeError. // Error returns the error message contained in the DecodeError.
@@ -100,7 +98,7 @@ func (e *DecodeError) Key() Key {
// //
//nolint:funlen //nolint:funlen
func wrapDecodeError(document []byte, de *unstable.ParserError) *DecodeError { func wrapDecodeError(document []byte, de *unstable.ParserError) *DecodeError {
offset := subsliceOffset(document, de.Highlight) offset := cap(document) - cap(de.Highlight)
errMessage := de.Error() errMessage := de.Error()
errLine, errColumn := positionAtEnd(document[:offset]) errLine, errColumn := positionAtEnd(document[:offset])
@@ -260,24 +258,5 @@ func positionAtEnd(b []byte) (row int, column int) {
} }
} }
return row, column return
}
// subsliceOffset returns the byte offset of subslice within data.
// subslice must share the same backing array as data.
func subsliceOffset(data []byte, subslice []byte) int {
if len(subslice) == 0 {
return 0
}
// Use reflect to get the data pointers of both slices.
// This is safe because we're only reading the pointer values for comparison.
dataPtr := reflect.ValueOf(data).Pointer()
subPtr := reflect.ValueOf(subslice).Pointer()
offset := int(subPtr - dataPtr)
if offset < 0 || offset > len(data) {
panic("subslice is not within data")
}
return offset
} }
+8 -89
View File
@@ -7,12 +7,13 @@ import (
"strings" "strings"
"testing" "testing"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
"git.ostiwe.com/ostiwe/go-toml/v2/unstable" "github.com/pelletier/go-toml/v2/unstable"
) )
//nolint:funlen //nolint:funlen
func TestDecodeError(t *testing.T) { func TestDecodeError(t *testing.T) {
examples := []struct { examples := []struct {
desc string desc string
doc [3]string doc [3]string
@@ -160,12 +161,13 @@ line 5`,
for _, e := range examples { for _, e := range examples {
e := e e := e
t.Run(e.desc, func(t *testing.T) { t.Run(e.desc, func(t *testing.T) {
b := bytes.Buffer{} b := bytes.Buffer{}
b.WriteString(e.doc[0]) b.Write([]byte(e.doc[0]))
start := b.Len() start := b.Len()
b.WriteString(e.doc[1]) b.Write([]byte(e.doc[1]))
end := b.Len() end := b.Len()
b.WriteString(e.doc[2]) b.Write([]byte(e.doc[2]))
doc := b.Bytes() doc := b.Bytes()
hl := doc[start:end] hl := doc[start:end]
@@ -187,6 +189,7 @@ line 5`,
} }
func TestDecodeError_Accessors(t *testing.T) { func TestDecodeError_Accessors(t *testing.T) {
e := DecodeError{ e := DecodeError{
message: "foo", message: "foo",
line: 1, line: 1,
@@ -202,90 +205,6 @@ func TestDecodeError_Accessors(t *testing.T) {
assert.Equal(t, "bar", e.String()) assert.Equal(t, "bar", e.String())
} }
func TestDecodeError_DuplicateContent(t *testing.T) {
// This test verifies that when the same content appears multiple times
// in the document, the error correctly points to the actual location
// of the error, not the first occurrence of the content.
//
// The document has "1__2" on line 1 and "3__4" on line 2.
// Both have "__" which is invalid, but we want to ensure errors
// on line 2 report line 2, not line 1.
doc := `a = 1
b = 3__4`
var v map[string]int
err := Unmarshal([]byte(doc), &v)
var derr *DecodeError
if !errors.As(err, &derr) {
t.Fatal("error not in expected format")
}
row, col := derr.Position()
// The error should be on line 2 where "3__4" is
if row != 2 {
t.Errorf("expected error on row 2, got row %d", row)
}
// Column should point to the "__" part (after "3")
if col < 5 {
t.Errorf("expected error at column >= 5, got column %d", col)
}
}
func TestDecodeError_Position(t *testing.T) {
// Test that error positions are correctly reported for various error locations
examples := []struct {
name string
doc string
expectedRow int
minCol int
}{
{
name: "error on first line",
doc: `a = 1__2`,
expectedRow: 1,
minCol: 5,
},
{
name: "error on second line",
doc: "a = 1\nb = 2__3",
expectedRow: 2,
minCol: 5,
},
{
name: "error on third line",
doc: "a = 1\nb = 2\nc = 3__4",
expectedRow: 3,
minCol: 5,
},
{
name: "missing equals on last line without trailing newline",
doc: "a = 1\nb = 2\nc",
expectedRow: 3,
minCol: 1,
},
}
for _, e := range examples {
t.Run(e.name, func(t *testing.T) {
var v map[string]int
err := Unmarshal([]byte(e.doc), &v)
var derr *DecodeError
if !errors.As(err, &derr) {
t.Fatal("error not in expected format")
}
row, col := derr.Position()
assert.Equal(t, e.expectedRow, row)
if col < e.minCol {
t.Errorf("expected column >= %d, got %d", e.minCol, col)
}
})
}
}
func TestStrictErrorUnwrap(t *testing.T) { func TestStrictErrorUnwrap(t *testing.T) {
fo := bytes.NewBufferString(` fo := bytes.NewBufferString(`
Missing = 1 Missing = 1
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"log" "log"
"strconv" "strconv"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
) )
type customInt int type customInt int
+2 -2
View File
@@ -3,8 +3,8 @@ package toml_test
import ( import (
"testing" "testing"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
func TestFastSimpleInt(t *testing.T) { func TestFastSimpleInt(t *testing.T) {
+3 -3
View File
@@ -5,14 +5,14 @@ import (
"strings" "strings"
"testing" "testing"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
func FuzzUnmarshal(f *testing.F) { func FuzzUnmarshal(f *testing.F) {
file, err := os.ReadFile("benchmark/benchmark.toml") file, err := os.ReadFile("benchmark/benchmark.toml")
if err != nil { if err != nil {
f.Error(err) panic(err)
} }
f.Add(file) f.Add(file)
+1 -1
View File
@@ -1,3 +1,3 @@
module git.ostiwe.com/ostiwe/go-toml/v2 module github.com/pelletier/go-toml/v2
go 1.21.0 go 1.21.0
+25 -31
View File
@@ -1,4 +1,3 @@
// Package assert provides assertion functions for unit testing.
package assert package assert
import ( import (
@@ -10,67 +9,66 @@ import (
) )
// True asserts that an expression is true. // True asserts that an expression is true.
func True(tb testing.TB, ok bool, msgAndArgs ...any) { func True(t testing.TB, ok bool, msgAndArgs ...any) {
tb.Helper()
if ok { if ok {
return return
} }
tb.Fatal(formatMsgAndArgs("Expected expression to be true", msgAndArgs...)) t.Helper()
t.Fatal(formatMsgAndArgs("Expected expression to be true", msgAndArgs...))
} }
// False asserts that an expression is false. // False asserts that an expression is false.
func False(tb testing.TB, ok bool, msgAndArgs ...any) { func False(t testing.TB, ok bool, msgAndArgs ...any) {
tb.Helper()
if !ok { if !ok {
return return
} }
tb.Fatal(formatMsgAndArgs("Expected expression to be false", msgAndArgs...)) t.Helper()
t.Fatal(formatMsgAndArgs("Expected expression to be false", msgAndArgs...))
} }
// Equal asserts that "expected" and "actual" are equal. // Equal asserts that "expected" and "actual" are equal.
func Equal[T any](tb testing.TB, expected, actual T, msgAndArgs ...any) { func Equal[T any](t testing.TB, expected, actual T, msgAndArgs ...any) {
tb.Helper()
if objectsAreEqual(expected, actual) { if objectsAreEqual(expected, actual) {
return return
} }
t.Helper()
msg := formatMsgAndArgs("Expected values to be equal:", msgAndArgs...) msg := formatMsgAndArgs("Expected values to be equal:", msgAndArgs...)
tb.Fatalf("%s\n%s", msg, diff(expected, actual)) t.Fatalf("%s\n%s", msg, diff(expected, actual))
} }
// Error asserts that an error is not nil. // Error asserts that an error is not nil.
func Error(tb testing.TB, err error, msgAndArgs ...any) { func Error(t testing.TB, err error, msgAndArgs ...any) {
tb.Helper()
if err != nil { if err != nil {
return return
} }
tb.Fatal(formatMsgAndArgs("Expected an error", msgAndArgs...)) t.Helper()
t.Fatal(formatMsgAndArgs("Expected an error", msgAndArgs...))
} }
// NoError asserts that an error is nil. // NoError asserts that an error is nil.
func NoError(tb testing.TB, err error, msgAndArgs ...any) { func NoError(t testing.TB, err error, msgAndArgs ...any) {
tb.Helper()
if err == nil { if err == nil {
return return
} }
t.Helper()
msg := formatMsgAndArgs("Unexpected error:", msgAndArgs...) msg := formatMsgAndArgs("Unexpected error:", msgAndArgs...)
tb.Fatalf("%s\n%+v", msg, err) t.Fatalf("%s\n%+v", msg, err)
} }
// Panics asserts that the given function panics. // Panics asserts that the given function panics.
func Panics(tb testing.TB, fn func(), msgAndArgs ...any) { func Panics(t testing.TB, fn func(), msgAndArgs ...any) {
tb.Helper() t.Helper()
defer func() { defer func() {
if recover() == nil { if recover() == nil {
msg := formatMsgAndArgs("Expected function to panic", msgAndArgs...) msg := formatMsgAndArgs("Expected function to panic", msgAndArgs...)
tb.Fatal(msg) t.Fatal(msg)
} }
}() }()
fn() fn()
} }
// Zero asserts that a value is its zero value. // Zero asserts that a value is its zero value.
func Zero[T any](tb testing.TB, value T, msgAndArgs ...any) { func Zero[T any](t testing.TB, value T, msgAndArgs ...any) {
tb.Helper()
var zero T var zero T
if objectsAreEqual(value, zero) { if objectsAreEqual(value, zero) {
return return
@@ -79,26 +77,22 @@ func Zero[T any](tb testing.TB, value T, msgAndArgs ...any) {
if (val.Kind() == reflect.Slice || val.Kind() == reflect.Map || val.Kind() == reflect.Array) && val.Len() == 0 { if (val.Kind() == reflect.Slice || val.Kind() == reflect.Map || val.Kind() == reflect.Array) && val.Len() == 0 {
return return
} }
t.Helper()
msg := formatMsgAndArgs("Expected zero value but got:", msgAndArgs...) msg := formatMsgAndArgs("Expected zero value but got:", msgAndArgs...)
tb.Fatalf("%s\n%v", msg, value) t.Fatalf("%s\n%v", msg, value)
} }
func NotZero[T any](tb testing.TB, value T, msgAndArgs ...any) { func NotZero[T any](t testing.TB, value T, msgAndArgs ...any) {
tb.Helper()
var zero T var zero T
if !objectsAreEqual(value, zero) { if !objectsAreEqual(value, zero) {
val := reflect.ValueOf(value) val := reflect.ValueOf(value)
switch val.Kind() { if !((val.Kind() == reflect.Slice || val.Kind() == reflect.Map || val.Kind() == reflect.Array) && val.Len() == 0) {
case reflect.Slice, reflect.Map, reflect.Array:
if val.Len() > 0 {
return
}
default:
return return
} }
} }
t.Helper()
msg := formatMsgAndArgs("Unexpected zero value:", msgAndArgs...) msg := formatMsgAndArgs("Unexpected zero value:", msgAndArgs...)
tb.Fatalf("%s\n%v", msg, value) t.Fatalf("%s\n%v", msg, value)
} }
func formatMsgAndArgs(msg string, args ...any) string { func formatMsgAndArgs(msg string, args ...any) string {
+66 -99
View File
@@ -1,7 +1,6 @@
package assert package assert
import ( import (
"errors"
"fmt" "fmt"
"testing" "testing"
) )
@@ -13,167 +12,135 @@ type Data struct {
func TestBadMessage(t *testing.T) { func TestBadMessage(t *testing.T) {
invalidMessage := func() { True(t, false, 1234) } invalidMessage := func() { True(t, false, 1234) }
assertOk(t, "Non-fmt message value", func(tb testing.TB) { assertOk(t, "Non-fmt message value", func(t testing.TB) {
tb.Helper() Panics(t, invalidMessage)
Panics(tb, invalidMessage)
}) })
assertFail(t, "Non-fmt message value", func(tb testing.TB) { assertFail(t, "Non-fmt message value", func(t testing.TB) {
tb.Helper() True(t, false, "example %s", "message")
True(tb, false, "example %s", "message")
}) })
} }
func TestTrue(t *testing.T) { func TestTrue(t *testing.T) {
assertOk(t, "Succeed", func(tb testing.TB) { assertOk(t, "Succeed", func(t testing.TB) {
tb.Helper() True(t, 1 > 0)
True(tb, 1 > 0)
}) })
assertFail(t, "Fail", func(tb testing.TB) { assertFail(t, "Fail", func(t testing.TB) {
tb.Helper() True(t, 1 < 0)
True(tb, 1 < 0)
}) })
} }
func TestFalse(t *testing.T) { func TestFalse(t *testing.T) {
assertOk(t, "Succeed", func(tb testing.TB) { assertOk(t, "Succeed", func(t testing.TB) {
tb.Helper() False(t, 1 < 0)
False(tb, 1 < 0)
}) })
assertFail(t, "Fail", func(tb testing.TB) { assertFail(t, "Fail", func(t testing.TB) {
tb.Helper() False(t, 1 > 0)
False(tb, 1 > 0)
}) })
} }
func TestEqual(t *testing.T) { func TestEqual(t *testing.T) {
assertOk(t, "Nil", func(tb testing.TB) { assertOk(t, "Nil", func(t testing.TB) {
tb.Helper() Equal(t, interface{}(nil), interface{}(nil))
Equal(tb, interface{}(nil), interface{}(nil))
}) })
assertOk(t, "Identical structs", func(tb testing.TB) { assertOk(t, "Identical structs", func(t testing.TB) {
tb.Helper() Equal(t, Data{"expected", 1234}, Data{"expected", 1234})
Equal(tb, Data{"expected", 1234}, Data{"expected", 1234})
}) })
assertFail(t, "Different structs", func(tb testing.TB) { assertFail(t, "Different structs", func(t testing.TB) {
tb.Helper() Equal(t, Data{"expected", 1234}, Data{"actual", 1234})
Equal(tb, Data{"expected", 1234}, Data{"actual", 1234})
}) })
assertOk(t, "Identical numbers", func(tb testing.TB) { assertOk(t, "Identical numbers", func(t testing.TB) {
tb.Helper() Equal(t, 1234, 1234)
Equal(tb, 1234, 1234)
}) })
assertFail(t, "Identical numbers", func(tb testing.TB) { assertFail(t, "Identical numbers", func(t testing.TB) {
tb.Helper() Equal(t, 1234, 1324)
Equal(tb, 1234, 1324)
}) })
assertOk(t, "Zero-length byte arrays", func(tb testing.TB) { assertOk(t, "Zero-length byte arrays", func(t testing.TB) {
tb.Helper() Equal(t, []byte(nil), []byte(""))
Equal(tb, []byte(nil), []byte(""))
}) })
assertOk(t, "Identical byte arrays", func(tb testing.TB) { assertOk(t, "Identical byte arrays", func(t testing.TB) {
tb.Helper() Equal(t, []byte{1, 2, 3, 4}, []byte{1, 2, 3, 4})
Equal(tb, []byte{1, 2, 3, 4}, []byte{1, 2, 3, 4})
}) })
assertFail(t, "Different byte arrays", func(tb testing.TB) { assertFail(t, "Different byte arrays", func(t testing.TB) {
tb.Helper() Equal(t, []byte{1, 2, 3, 4}, []byte{1, 3, 2, 4})
Equal(tb, []byte{1, 2, 3, 4}, []byte{1, 3, 2, 4})
}) })
assertOk(t, "Identical strings", func(tb testing.TB) { assertOk(t, "Identical strings", func(t testing.TB) {
tb.Helper() Equal(t, "example", "example")
Equal(tb, "example", "example")
}) })
assertFail(t, "Identical strings", func(tb testing.TB) { assertFail(t, "Identical strings", func(t testing.TB) {
tb.Helper() Equal(t, "example", "elpmaxe")
Equal(tb, "example", "elpmaxe")
}) })
} }
func TestError(t *testing.T) { func TestError(t *testing.T) {
assertOk(t, "Error", func(tb testing.TB) { assertOk(t, "Error", func(t testing.TB) {
tb.Helper() Error(t, fmt.Errorf("example"))
Error(tb, errors.New("example"))
}) })
assertFail(t, "Nil", func(tb testing.TB) { assertFail(t, "Nil", func(t testing.TB) {
tb.Helper() Error(t, nil)
Error(tb, nil)
}) })
} }
func TestNoError(t *testing.T) { func TestNoError(t *testing.T) {
assertFail(t, "Error", func(tb testing.TB) { assertFail(t, "Error", func(t testing.TB) {
tb.Helper() NoError(t, fmt.Errorf("example"))
NoError(tb, errors.New("example"))
}) })
assertOk(t, "Nil", func(tb testing.TB) { assertOk(t, "Nil", func(t testing.TB) {
tb.Helper() NoError(t, nil)
NoError(tb, nil)
}) })
} }
func TestPanics(t *testing.T) { func TestPanics(t *testing.T) {
willPanic := func() { panic("example") } willPanic := func() { panic("example") }
wontPanic := func() {} wontPanic := func() {}
assertOk(t, "Will panic", func(tb testing.TB) { assertOk(t, "Will panic", func(t testing.TB) {
tb.Helper() Panics(t, willPanic)
Panics(tb, willPanic)
}) })
assertFail(t, "Won't panic", func(tb testing.TB) { assertFail(t, "Won't panic", func(t testing.TB) {
tb.Helper() Panics(t, wontPanic)
Panics(tb, wontPanic)
}) })
} }
func TestZero(t *testing.T) { func TestZero(t *testing.T) {
assertOk(t, "Empty struct", func(tb testing.TB) { assertOk(t, "Empty struct", func(t testing.TB) {
tb.Helper() Zero(t, Data{})
Zero(tb, Data{})
}) })
assertFail(t, "Non-empty struct", func(tb testing.TB) { assertFail(t, "Non-empty struct", func(t testing.TB) {
tb.Helper() Zero(t, Data{Label: "example"})
Zero(tb, Data{Label: "example"})
}) })
assertOk(t, "Nil slice", func(tb testing.TB) { assertOk(t, "Nil slice", func(t testing.TB) {
tb.Helper()
var slice []int var slice []int
Zero(tb, slice) Zero(t, slice)
}) })
assertFail(t, "Non-empty slice", func(tb testing.TB) { assertFail(t, "Non-empty slice", func(t testing.TB) {
tb.Helper()
slice := []int{1, 2, 3, 4} slice := []int{1, 2, 3, 4}
Zero(tb, slice) Zero(t, slice)
}) })
assertOk(t, "Zero-length slice", func(tb testing.TB) { assertOk(t, "Zero-length slice", func(t testing.TB) {
tb.Helper()
slice := []int{} slice := []int{}
Zero(tb, slice) Zero(t, slice)
}) })
} }
func TestNotZero(t *testing.T) { func TestNotZero(t *testing.T) {
assertFail(t, "Empty struct", func(tb testing.TB) { assertFail(t, "Empty struct", func(t testing.TB) {
tb.Helper()
zero := Data{} zero := Data{}
NotZero(tb, zero) NotZero(t, zero)
}) })
assertOk(t, "Non-empty struct", func(tb testing.TB) { assertOk(t, "Non-empty struct", func(t testing.TB) {
tb.Helper()
notZero := Data{Label: "example"} notZero := Data{Label: "example"}
NotZero(tb, notZero) NotZero(t, notZero)
}) })
assertFail(t, "Nil slice", func(tb testing.TB) { assertFail(t, "Nil slice", func(t testing.TB) {
tb.Helper()
var slice []int var slice []int
NotZero(tb, slice) NotZero(t, slice)
}) })
assertFail(t, "Zero-length slice", func(tb testing.TB) { assertFail(t, "Zero-length slice", func(t testing.TB) {
tb.Helper()
slice := []int{} slice := []int{}
NotZero(tb, slice) NotZero(t, slice)
}) })
assertOk(t, "Non-empty slice", func(tb testing.TB) { assertOk(t, "Non-empty slice", func(t testing.TB) {
tb.Helper()
slice := []int{1, 2, 3, 4} slice := []int{1, 2, 3, 4}
NotZero(tb, slice) NotZero(t, slice)
}) })
} }
@@ -190,7 +157,7 @@ func (t *testCase) Fatalf(message string, args ...interface{}) {
t.failed = fmt.Sprintf(message, args...) t.failed = fmt.Sprintf(message, args...)
} }
func assertFail(t *testing.T, name string, fn func(testing.TB)) { func assertFail(t *testing.T, name string, fn func(t testing.TB)) {
t.Helper() t.Helper()
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
t.Helper() t.Helper()
@@ -204,7 +171,7 @@ func assertFail(t *testing.T, name string, fn func(testing.TB)) {
}) })
} }
func assertOk(t *testing.T, name string, fn func(testing.TB)) { func assertOk(t *testing.T, name string, fn func(t testing.TB)) {
t.Helper() t.Helper()
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
t.Helper() t.Helper()
+3 -3
View File
@@ -1,6 +1,6 @@
package characters package characters
var invalidASCIITable = [256]bool{ var invalidAsciiTable = [256]bool{
0x00: true, 0x00: true,
0x01: true, 0x01: true,
0x02: true, 0x02: true,
@@ -37,6 +37,6 @@ var invalidASCIITable = [256]bool{
0x7F: true, 0x7F: true,
} }
func InvalidASCII(b byte) bool { func InvalidAscii(b byte) bool {
return invalidASCIITable[b] return invalidAsciiTable[b]
} }
+46 -22
View File
@@ -1,12 +1,20 @@
// Package characters provides functions for working with string encodings.
package characters package characters
import ( import (
"unicode/utf8" "unicode/utf8"
) )
// Utf8TomlValidAlreadyEscaped verifies that a given string is only made of type utf8Err struct {
// valid UTF-8 characters allowed by the TOML spec: Index int
Size int
}
func (u utf8Err) Zero() bool {
return u.Size == 0
}
// Verified that a given string is only made of valid UTF-8 characters allowed
// by the TOML spec:
// //
// Any Unicode character may be used except those that must be escaped: // Any Unicode character may be used except those that must be escaped:
// quotation mark, backslash, and the control characters other than tab (U+0000 // quotation mark, backslash, and the control characters other than tab (U+0000
@@ -15,8 +23,8 @@ import (
// It is a copy of the Go 1.17 utf8.Valid implementation, tweaked to exit early // It is a copy of the Go 1.17 utf8.Valid implementation, tweaked to exit early
// when a character is not allowed. // when a character is not allowed.
// //
// The returned slice is empty if the string is valid, or contains the bytes // The returned utf8Err is Zero() if the string is valid, or contains the byte
// of the invalid character. // index and size of the invalid character.
// //
// quotation mark => already checked // quotation mark => already checked
// backslash => already checked // backslash => already checked
@@ -24,8 +32,9 @@ import (
// 0x9 => tab, ok // 0x9 => tab, ok
// 0xA - 0x1F => invalid // 0xA - 0x1F => invalid
// 0x7F => invalid // 0x7F => invalid
func Utf8TomlValidAlreadyEscaped(p []byte) []byte { func Utf8TomlValidAlreadyEscaped(p []byte) (err utf8Err) {
// Fast path. Check for and skip 8 bytes of ASCII characters per iteration. // Fast path. Check for and skip 8 bytes of ASCII characters per iteration.
offset := 0
for len(p) >= 8 { for len(p) >= 8 {
// Combining two 32 bit loads allows the same code to be used // Combining two 32 bit loads allows the same code to be used
// for 32 and 64 bit platforms. // for 32 and 64 bit platforms.
@@ -39,19 +48,24 @@ func Utf8TomlValidAlreadyEscaped(p []byte) []byte {
} }
for i, b := range p[:8] { for i, b := range p[:8] {
if InvalidASCII(b) { if InvalidAscii(b) {
return p[i : i+1] err.Index = offset + i
err.Size = 1
return
} }
} }
p = p[8:] p = p[8:]
offset += 8
} }
n := len(p) n := len(p)
for i := 0; i < n; { for i := 0; i < n; {
pi := p[i] pi := p[i]
if pi < utf8.RuneSelf { if pi < utf8.RuneSelf {
if InvalidASCII(pi) { if InvalidAscii(pi) {
return p[i : i+1] err.Index = offset + i
err.Size = 1
return
} }
i++ i++
continue continue
@@ -59,34 +73,44 @@ func Utf8TomlValidAlreadyEscaped(p []byte) []byte {
x := first[pi] x := first[pi]
if x == xx { if x == xx {
// Illegal starter byte. // Illegal starter byte.
return p[i : i+1] err.Index = offset + i
err.Size = 1
return
} }
size := int(x & 7) size := int(x & 7)
if i+size > n { if i+size > n {
// Short or invalid. // Short or invalid.
return p[i:n] err.Index = offset + i
err.Size = n - i
return
} }
accept := acceptRanges[x>>4] accept := acceptRanges[x>>4]
if c := p[i+1]; c < accept.lo || accept.hi < c { if c := p[i+1]; c < accept.lo || accept.hi < c {
return p[i : i+2] err.Index = offset + i
} else if size == 2 { //revive:disable:empty-block err.Size = 2
return
} else if size == 2 {
} else if c := p[i+2]; c < locb || hicb < c { } else if c := p[i+2]; c < locb || hicb < c {
return p[i : i+3] err.Index = offset + i
} else if size == 3 { //revive:disable:empty-block err.Size = 3
return
} else if size == 3 {
} else if c := p[i+3]; c < locb || hicb < c { } else if c := p[i+3]; c < locb || hicb < c {
return p[i : i+4] err.Index = offset + i
err.Size = 4
return
} }
i += size i += size
} }
return nil return
} }
// Utf8ValidNext returns the size of the next rune if valid, 0 otherwise. // Return the size of the next rune if valid, 0 otherwise.
func Utf8ValidNext(p []byte) int { func Utf8ValidNext(p []byte) int {
c := p[0] c := p[0]
if c < utf8.RuneSelf { if c < utf8.RuneSelf {
if InvalidASCII(c) { if InvalidAscii(c) {
return 0 return 0
} }
return 1 return 1
@@ -105,10 +129,10 @@ func Utf8ValidNext(p []byte) int {
accept := acceptRanges[x>>4] accept := acceptRanges[x>>4]
if c := p[1]; c < accept.lo || accept.hi < c { if c := p[1]; c < accept.lo || accept.hi < c {
return 0 return 0
} else if size == 2 { //nolint:revive } else if size == 2 {
} else if c := p[2]; c < locb || hicb < c { } else if c := p[2]; c < locb || hicb < c {
return 0 return 0
} else if size == 3 { //nolint:revive } else if size == 3 {
} else if c := p[3]; c < locb || hicb < c { } else if c := p[3]; c < locb || hicb < c {
return 0 return 0
} }
+9 -9
View File
@@ -1,4 +1,3 @@
// Package cli provides common functions for command-line programs.
package cli package cli
import ( import (
@@ -9,7 +8,7 @@ import (
"io" "io"
"os" "os"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
) )
type ConvertFn func(r io.Reader, w io.Writer) error type ConvertFn func(r io.Reader, w io.Writer) error
@@ -28,16 +27,17 @@ func (p *Program) Execute() {
os.Exit(p.main(flag.Args(), os.Stdin, os.Stdout, os.Stderr)) os.Exit(p.main(flag.Args(), os.Stdin, os.Stdout, os.Stderr))
} }
func (p *Program) main(files []string, input io.Reader, output, stderr io.Writer) int { func (p *Program) main(files []string, input io.Reader, output, error io.Writer) int {
err := p.run(files, input, output) err := p.run(files, input, output)
if err != nil { if err != nil {
var derr *toml.DecodeError var derr *toml.DecodeError
if errors.As(err, &derr) { if errors.As(err, &derr) {
_, _ = fmt.Fprintln(stderr, derr.String()) fmt.Fprintln(error, derr.String())
row, col := derr.Position() row, col := derr.Position()
_, _ = fmt.Fprintln(stderr, "error occurred at row", row, "column", col) fmt.Fprintln(error, "error occurred at row", row, "column", col)
} else { } else {
_, _ = fmt.Fprintln(stderr, err.Error()) fmt.Fprintln(error, err.Error())
} }
return -1 return -1
@@ -54,7 +54,7 @@ func (p *Program) run(files []string, input io.Reader, output io.Writer) error {
if err != nil { if err != nil {
return err return err
} }
defer func() { _ = f.Close() }() defer f.Close()
input = f input = f
} }
return p.Fn(input, output) return p.Fn(input, output)
@@ -71,7 +71,7 @@ func (p *Program) runAllFilesInPlace(files []string) error {
} }
func (p *Program) runFileInPlace(path string) error { func (p *Program) runFileInPlace(path string) error {
in, err := os.ReadFile(path) // #nosec G304 in, err := os.ReadFile(path)
if err != nil { if err != nil {
return err return err
} }
@@ -83,5 +83,5 @@ func (p *Program) runFileInPlace(path string) error {
return err return err
} }
return os.WriteFile(path, out.Bytes(), 0o600) return os.WriteFile(path, out.Bytes(), 0600)
} }
+22 -18
View File
@@ -2,15 +2,15 @@ package cli
import ( import (
"bytes" "bytes"
"errors" "fmt"
"io" "io"
"os" "os"
"path" "path"
"strings" "strings"
"testing" "testing"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
func processMain(args []string, input io.Reader, stdout, stderr io.Writer, f ConvertFn) int { func processMain(args []string, input io.Reader, stdout, stderr io.Writer, f ConvertFn) int {
@@ -23,7 +23,7 @@ func TestProcessMainStdin(t *testing.T) {
stderr := new(bytes.Buffer) stderr := new(bytes.Buffer)
input := strings.NewReader("this is the input") input := strings.NewReader("this is the input")
exit := processMain([]string{}, input, stdout, stderr, func(io.Reader, io.Writer) error { exit := processMain([]string{}, input, stdout, stderr, func(r io.Reader, w io.Writer) error {
return nil return nil
}) })
@@ -37,8 +37,8 @@ func TestProcessMainStdinErr(t *testing.T) {
stderr := new(bytes.Buffer) stderr := new(bytes.Buffer)
input := strings.NewReader("this is the input") input := strings.NewReader("this is the input")
exit := processMain([]string{}, input, stdout, stderr, func(io.Reader, io.Writer) error { exit := processMain([]string{}, input, stdout, stderr, func(r io.Reader, w io.Writer) error {
return errors.New("something bad") return fmt.Errorf("something bad")
}) })
assert.Equal(t, -1, exit) assert.Equal(t, -1, exit)
@@ -51,7 +51,7 @@ func TestProcessMainStdinDecodeErr(t *testing.T) {
stderr := new(bytes.Buffer) stderr := new(bytes.Buffer)
input := strings.NewReader("this is the input") input := strings.NewReader("this is the input")
exit := processMain([]string{}, input, stdout, stderr, func(io.Reader, io.Writer) error { exit := processMain([]string{}, input, stdout, stderr, func(r io.Reader, w io.Writer) error {
var v interface{} var v interface{}
return toml.Unmarshal([]byte(`qwe = 001`), &v) return toml.Unmarshal([]byte(`qwe = 001`), &v)
}) })
@@ -62,16 +62,16 @@ func TestProcessMainStdinDecodeErr(t *testing.T) {
} }
func TestProcessMainFileExists(t *testing.T) { func TestProcessMainFileExists(t *testing.T) {
tmpfile, err := os.CreateTemp(t.TempDir(), "example") tmpfile, err := os.CreateTemp("", "example")
assert.NoError(t, err) assert.NoError(t, err)
_, err = tmpfile.WriteString(`some data`) defer os.Remove(tmpfile.Name())
_, err = tmpfile.Write([]byte(`some data`))
assert.NoError(t, err) assert.NoError(t, err)
assert.NoError(t, tmpfile.Close())
stdout := new(bytes.Buffer) stdout := new(bytes.Buffer)
stderr := new(bytes.Buffer) stderr := new(bytes.Buffer)
exit := processMain([]string{tmpfile.Name()}, nil, stdout, stderr, func(io.Reader, io.Writer) error { exit := processMain([]string{tmpfile.Name()}, nil, stdout, stderr, func(r io.Reader, w io.Writer) error {
return nil return nil
}) })
@@ -84,7 +84,7 @@ func TestProcessMainFileDoesNotExist(t *testing.T) {
stdout := new(bytes.Buffer) stdout := new(bytes.Buffer)
stderr := new(bytes.Buffer) stderr := new(bytes.Buffer)
exit := processMain([]string{"/lets/hope/this/does/not/exist"}, nil, stdout, stderr, func(io.Reader, io.Writer) error { exit := processMain([]string{"/lets/hope/this/does/not/exist"}, nil, stdout, stderr, func(r io.Reader, w io.Writer) error {
return nil return nil
}) })
@@ -94,14 +94,16 @@ func TestProcessMainFileDoesNotExist(t *testing.T) {
} }
func TestProcessMainFilesInPlace(t *testing.T) { func TestProcessMainFilesInPlace(t *testing.T) {
dir := t.TempDir() dir, err := os.MkdirTemp("", "")
assert.NoError(t, err)
defer os.RemoveAll(dir)
path1 := path.Join(dir, "file1") path1 := path.Join(dir, "file1")
path2 := path.Join(dir, "file2") path2 := path.Join(dir, "file2")
err := os.WriteFile(path1, []byte("content 1"), 0o600) err = os.WriteFile(path1, []byte("content 1"), 0600)
assert.NoError(t, err) assert.NoError(t, err)
err = os.WriteFile(path2, []byte("content 2"), 0o600) err = os.WriteFile(path2, []byte("content 2"), 0600)
assert.NoError(t, err) assert.NoError(t, err)
p := Program{ p := Program{
@@ -134,15 +136,17 @@ func TestProcessMainFilesInPlaceErrRead(t *testing.T) {
} }
func TestProcessMainFilesInPlaceFailFn(t *testing.T) { func TestProcessMainFilesInPlaceFailFn(t *testing.T) {
dir := t.TempDir() dir, err := os.MkdirTemp("", "")
assert.NoError(t, err)
defer os.RemoveAll(dir)
path1 := path.Join(dir, "file1") path1 := path.Join(dir, "file1")
err := os.WriteFile(path1, []byte("content 1"), 0o600) err = os.WriteFile(path1, []byte("content 1"), 0600)
assert.NoError(t, err) assert.NoError(t, err)
p := Program{ p := Program{
Fn: func(io.Reader, io.Writer) error { return errors.New("oh no") }, Fn: func(io.Reader, io.Writer) error { return fmt.Errorf("oh no") },
Inplace: true, Inplace: true,
} }
@@ -1,4 +1,4 @@
package imported_tests //revive:disable:var-naming package imported_tests
// Those tests have been imported from v1, but adjust to match the new // Those tests have been imported from v1, but adjust to match the new
// defaults of v2. // defaults of v2.
@@ -8,8 +8,8 @@ import (
"testing" "testing"
"time" "time"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
func TestDocMarshal(t *testing.T) { func TestDocMarshal(t *testing.T) {
@@ -21,12 +21,12 @@ func TestDocMarshal(t *testing.T) {
Subdocs testDocSubs `toml:"subdoc"` Subdocs testDocSubs `toml:"subdoc"`
Basics testDocBasics `toml:"basic"` Basics testDocBasics `toml:"basic"`
SubDocList []testSubDoc `toml:"subdoclist"` SubDocList []testSubDoc `toml:"subdoclist"`
err int `toml:"shouldntBeHere"` //nolint:unused err int `toml:"shouldntBeHere"`
unexported int `toml:"shouldntBeHere"` unexported int `toml:"shouldntBeHere"`
Unexported2 int `toml:"-"` Unexported2 int `toml:"-"`
} }
docData := testDoc{ var docData = testDoc{
Title: "TOML Marshal Testing", Title: "TOML Marshal Testing",
unexported: 0, unexported: 0,
Unexported2: 0, Unexported2: 0,
@@ -128,7 +128,8 @@ String2 = 'Two'
String2 = 'Three' String2 = 'Three'
` `
assert.Equal(t, expected, string(result)) assert.Equal(t, string(expected), string(result))
} }
func TestEmptyMarshal(t *testing.T) { func TestEmptyMarshal(t *testing.T) {
@@ -163,7 +164,7 @@ stringlist = []
[map] [map]
` `
assert.Equal(t, expected, string(result)) assert.Equal(t, string(expected), string(result))
} }
type textMarshaler struct { type textMarshaler struct {
@@ -1,4 +1,4 @@
package imported_tests //revive:disable:var-naming package imported_tests
// Those tests were imported directly from go-toml v1 // Those tests were imported directly from go-toml v1
// https://raw.githubusercontent.com/pelletier/go-toml/a2e52561804c6cd9392ebf0048ca64fe4af67a43/marshal_test.go // https://raw.githubusercontent.com/pelletier/go-toml/a2e52561804c6cd9392ebf0048ca64fe4af67a43/marshal_test.go
@@ -15,8 +15,8 @@ import (
"testing" "testing"
"time" "time"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
type basicMarshalTestStruct struct { type basicMarshalTestStruct struct {
@@ -149,6 +149,9 @@ type quotedKeyMarshalTestStruct struct {
SubList []basicMarshalTestSubStruct `toml:"W.sublist-𝟘"` SubList []basicMarshalTestSubStruct `toml:"W.sublist-𝟘"`
} }
// TODO: Remove nolint once var is used by a test
//
//nolint:deadcode,unused,varcheck
var quotedKeyMarshalTestData = quotedKeyMarshalTestStruct{ var quotedKeyMarshalTestData = quotedKeyMarshalTestStruct{
String: "Hello", String: "Hello",
Float: 3.5, Float: 3.5,
@@ -158,7 +161,7 @@ var quotedKeyMarshalTestData = quotedKeyMarshalTestStruct{
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var quotedKeyMarshalTestToml = []byte(`"Yfloat-𝟘" = 3.5 var quotedKeyMarshalTestToml = []byte(`"Yfloat-𝟘" = 3.5
"Z.string-àéù" = "Hello" "Z.string-àéù" = "Hello"
@@ -180,12 +183,11 @@ type testDoc struct {
Subdocs testDocSubs `toml:"subdoc"` Subdocs testDocSubs `toml:"subdoc"`
Basics testDocBasics `toml:"basic"` Basics testDocBasics `toml:"basic"`
SubDocList []testSubDoc `toml:"subdoclist"` SubDocList []testSubDoc `toml:"subdoclist"`
err int `toml:"shouldntBeHere"` //nolint:unused err int `toml:"shouldntBeHere"` // nolint:structcheck,unused
unexported int `toml:"shouldntBeHere"` unexported int `toml:"shouldntBeHere"`
Unexported2 int `toml:"-"` Unexported2 int `toml:"-"`
} }
//nolint:unused
type testMapDoc struct { type testMapDoc struct {
Title string `toml:"title"` Title string `toml:"title"`
BasicMap map[string]string `toml:"basic_map"` BasicMap map[string]string `toml:"basic_map"`
@@ -272,7 +274,7 @@ var docData = testDoc{
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var mapTestDoc = testMapDoc{ var mapTestDoc = testMapDoc{
Title: "TOML Marshal Testing", Title: "TOML Marshal Testing",
BasicMap: map[string]string{ BasicMap: map[string]string{
@@ -541,35 +543,31 @@ func TestNestedUnmarshal(t *testing.T) {
assert.Equal(t, nestedTestData, result) assert.Equal(t, nestedTestData, result)
} }
//nolint:unused
type customMarshalerParent struct { type customMarshalerParent struct {
Self customMarshaler `toml:"me"` Self customMarshaler `toml:"me"`
Friends []customMarshaler `toml:"friends"` Friends []customMarshaler `toml:"friends"`
} }
//nolint:unused
type customMarshaler struct { type customMarshaler struct {
FirstName string FirstName string
LastName string LastName string
} }
//nolint:unused
func (c customMarshaler) MarshalTOML() ([]byte, error) { func (c customMarshaler) MarshalTOML() ([]byte, error) {
fullName := fmt.Sprintf("%s %s", c.FirstName, c.LastName) fullName := fmt.Sprintf("%s %s", c.FirstName, c.LastName)
return []byte(fullName), nil return []byte(fullName), nil
} }
//nolint:unused
var customMarshalerData = customMarshaler{FirstName: "Sally", LastName: "Fields"} var customMarshalerData = customMarshaler{FirstName: "Sally", LastName: "Fields"}
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var customMarshalerToml = []byte(`Sally Fields`) var customMarshalerToml = []byte(`Sally Fields`)
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var nestedCustomMarshalerData = customMarshalerParent{ var nestedCustomMarshalerData = customMarshalerParent{
Self: customMarshaler{FirstName: "Maiku", LastName: "Suteda"}, Self: customMarshaler{FirstName: "Maiku", LastName: "Suteda"},
Friends: []customMarshaler{customMarshalerData}, Friends: []customMarshaler{customMarshalerData},
@@ -577,7 +575,7 @@ var nestedCustomMarshalerData = customMarshalerParent{
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var nestedCustomMarshalerToml = []byte(`friends = ["Sally Fields"] var nestedCustomMarshalerToml = []byte(`friends = ["Sally Fields"]
me = "Maiku Suteda" me = "Maiku Suteda"
`) `)
@@ -592,7 +590,7 @@ func (x *IntOrString) MarshalTOML() ([]byte, error) {
s := *(*string)(x) s := *(*string)(x)
_, err := strconv.Atoi(s) _, err := strconv.Atoi(s)
if err != nil { if err != nil {
return []byte(fmt.Sprintf(`"%s"`, s)), nil //nolint:nilerr return []byte(fmt.Sprintf(`"%s"`, s)), nil
} }
return []byte(s), nil return []byte(s), nil
} }
@@ -664,7 +662,7 @@ func (m *textPointerMarshaler) MarshalText() ([]byte, error) {
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var commentTestToml = []byte(` var commentTestToml = []byte(`
# it's a comment on type # it's a comment on type
[postgres] [postgres]
@@ -689,7 +687,6 @@ var commentTestToml = []byte(`
My = "Baar" My = "Baar"
`) `)
//nolint:unused
type mapsTestStruct struct { type mapsTestStruct struct {
Simple map[string]string Simple map[string]string
Paths map[string]string Paths map[string]string
@@ -703,7 +700,7 @@ type mapsTestStruct struct {
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var mapsTestData = mapsTestStruct{ var mapsTestData = mapsTestStruct{
Simple: map[string]string{ Simple: map[string]string{
"one plus one": "two", "one plus one": "two",
@@ -727,7 +724,7 @@ var mapsTestData = mapsTestStruct{
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var mapsTestToml = []byte(` var mapsTestToml = []byte(`
[Other] [Other]
"testing" = 3.9999 "testing" = 3.9999
@@ -750,7 +747,7 @@ var mapsTestToml = []byte(`
// TODO: Remove nolint once type is used by a test // TODO: Remove nolint once type is used by a test
// //
//nolint:unused //nolint:deadcode,unused
type structArrayNoTag struct { type structArrayNoTag struct {
A struct { A struct {
B []int64 B []int64
@@ -760,7 +757,7 @@ type structArrayNoTag struct {
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var customTagTestToml = []byte(` var customTagTestToml = []byte(`
[postgres] [postgres]
password = "bvalue" password = "bvalue"
@@ -775,7 +772,7 @@ var customTagTestToml = []byte(`
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var customCommentTagTestToml = []byte(` var customCommentTagTestToml = []byte(`
# db connection # db connection
[postgres] [postgres]
@@ -789,7 +786,7 @@ var customCommentTagTestToml = []byte(`
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var customCommentedTagTestToml = []byte(` var customCommentedTagTestToml = []byte(`
[postgres] [postgres]
# password = "bvalue" # password = "bvalue"
@@ -844,7 +841,7 @@ func TestUnmarshalTabInStringAndQuotedKey(t *testing.T) {
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var customMultilineTagTestToml = []byte(`int_slice = [ var customMultilineTagTestToml = []byte(`int_slice = [
1, 1,
2, 2,
@@ -854,7 +851,7 @@ var customMultilineTagTestToml = []byte(`int_slice = [
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var testDocBasicToml = []byte(` var testDocBasicToml = []byte(`
[document] [document]
bool_val = true bool_val = true
@@ -865,12 +862,16 @@ var testDocBasicToml = []byte(`
uint_val = 5001 uint_val = 5001
`) `)
//nolint:unused // TODO: Remove nolint once type is used by a test
//
//nolint:deadcode
type testDocCustomTag struct { type testDocCustomTag struct {
Doc testDocBasicsCustomTag `file:"document"` Doc testDocBasicsCustomTag `file:"document"`
} }
//nolint:unused // TODO: Remove nolint once type is used by a test
//
//nolint:deadcode
type testDocBasicsCustomTag struct { type testDocBasicsCustomTag struct {
Bool bool `file:"bool_val"` Bool bool `file:"bool_val"`
Date time.Time `file:"date_val"` Date time.Time `file:"date_val"`
@@ -881,7 +882,9 @@ type testDocBasicsCustomTag struct {
unexported int `file:"shouldntBeHere"` unexported int `file:"shouldntBeHere"`
} }
//nolint:unused // TODO: Remove nolint once var is used by a test
//
//nolint:deadcode,varcheck
var testDocCustomTagData = testDocCustomTag{ var testDocCustomTagData = testDocCustomTag{
Doc: testDocBasicsCustomTag{ Doc: testDocBasicsCustomTag{
Bool: true, Bool: true,
@@ -984,13 +987,13 @@ func TestUnmarshalInvalidPointerKind(t *testing.T) {
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused
type testDuration struct { type testDuration struct {
Nanosec time.Duration `toml:"nanosec"` Nanosec time.Duration `toml:"nanosec"`
Microsec1 time.Duration `toml:"microsec1"` Microsec1 time.Duration `toml:"microsec1"`
Microsec2 *time.Duration `toml:"microsec2"` Microsec2 *time.Duration `toml:"microsec2"`
Millisec time.Duration `toml:"millisec"` Millisec time.Duration `toml:"millisec"`
Sec time.Duration `toml:"sec"` //nolint:staticcheck Sec time.Duration `toml:"sec"`
Min time.Duration `toml:"min"` Min time.Duration `toml:"min"`
Hour time.Duration `toml:"hour"` Hour time.Duration `toml:"hour"`
Mixed time.Duration `toml:"mixed"` Mixed time.Duration `toml:"mixed"`
@@ -999,7 +1002,7 @@ type testDuration struct {
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var testDurationToml = []byte(` var testDurationToml = []byte(`
nanosec = "1ns" nanosec = "1ns"
microsec1 = "1us" microsec1 = "1us"
@@ -1014,7 +1017,7 @@ a_string = "15s"
// TODO: Remove nolint once var is used by a test // TODO: Remove nolint once var is used by a test
// //
//nolint:unused //nolint:deadcode,unused,varcheck
var testDurationToml2 = []byte(`a_string = "15s" var testDurationToml2 = []byte(`a_string = "15s"
hour = "1h0m0s" hour = "1h0m0s"
microsec1 = "1µs" microsec1 = "1µs"
@@ -1028,14 +1031,15 @@ sec = "1s"
// TODO: Remove nolint once type is used by a test // TODO: Remove nolint once type is used by a test
// //
//nolint:unused //nolint:deadcode,unused
type testBadDuration struct { type testBadDuration struct {
Val time.Duration `toml:"val"` Val time.Duration `toml:"val"`
} }
// TODO: add back camelCase test // TODO: add back camelCase test
var testCamelCaseKeyToml = []byte(`fooBar = 10`) var testCamelCaseKeyToml = []byte(`fooBar = 10`) //nolint:unused
//nolint:unused
func TestUnmarshalCamelCaseKey(t *testing.T) { func TestUnmarshalCamelCaseKey(t *testing.T) {
t.Skipf("don't know if it is a good idea to automatically convert like that yet") t.Skipf("don't know if it is a good idea to automatically convert like that yet")
var x struct { var x struct {
@@ -1054,7 +1058,7 @@ func TestUnmarshalCamelCaseKey(t *testing.T) {
func TestUnmarshalNegativeUint(t *testing.T) { func TestUnmarshalNegativeUint(t *testing.T) {
t.Skipf("not sure if we this should always error") t.Skipf("not sure if we this should always error")
type check struct{ U uint } type check struct{ U uint } // nolint:unused
err := toml.Unmarshal([]byte("U = -1"), &check{}) err := toml.Unmarshal([]byte("U = -1"), &check{})
assert.Error(t, err) assert.Error(t, err)
} }
@@ -1531,7 +1535,7 @@ func TestUnmarshalLocalDateTime(t *testing.T) {
} }
for i, example := range examples { for i, example := range examples {
doc := "date = " + example.in doc := fmt.Sprintf(`date = %s`, example.in)
t.Run(fmt.Sprintf("ToLocalDateTime_%d_%s", i, example.name), func(t *testing.T) { t.Run(fmt.Sprintf("ToLocalDateTime_%d_%s", i, example.name), func(t *testing.T) {
type dateStruct struct { type dateStruct struct {
@@ -1617,7 +1621,7 @@ func TestUnmarshalLocalTime(t *testing.T) {
} }
for i, example := range examples { for i, example := range examples {
doc := "Time = " + example.in doc := fmt.Sprintf(`Time = %s`, example.in)
t.Run(fmt.Sprintf("ToLocalTime_%d_%s", i, example.name), func(t *testing.T) { t.Run(fmt.Sprintf("ToLocalTime_%d_%s", i, example.name), func(t *testing.T) {
type dateStruct struct { type dateStruct struct {
@@ -1902,12 +1906,19 @@ func TestUnmarshalMixedTypeSlice(t *testing.T) {
ArrayField []interface{} ArrayField []interface{}
} }
//doc := []byte(`ArrayField = [3.14,100,true,"hello world",{Field = "inner1"},[{Field = "inner2"},{Field = "inner3"}]]
//`)
doc := []byte(`ArrayField = [{Field = "inner1"},[{Field = "inner2"},{Field = "inner3"}]] doc := []byte(`ArrayField = [{Field = "inner1"},[{Field = "inner2"},{Field = "inner3"}]]
`) `)
actual := TestStruct{} actual := TestStruct{}
expected := TestStruct{ expected := TestStruct{
ArrayField: []interface{}{ ArrayField: []interface{}{
//3.14,
//int64(100),
//true,
//"hello world",
map[string]interface{}{ map[string]interface{}{
"Field": "inner1", "Field": "inner1",
}, },
@@ -1993,10 +2004,9 @@ func TestDecoderStrict(t *testing.T) {
"Expected a *toml.StrictMissingError, got: %v", reflect.TypeOf(err), "Expected a *toml.StrictMissingError, got: %v", reflect.TypeOf(err),
) )
var se *toml.StrictMissingError se := err.(*toml.StrictMissingError)
assert.True(t, errors.As(err, &se))
keys := make([]toml.Key, 0, len(se.Errors)) keys := []toml.Key{}
for _, e := range se.Errors { for _, e := range se.Errors {
keys = append(keys, e.Key()) keys = append(keys, e.Key())
@@ -2016,7 +2026,6 @@ func TestDecoderStrict(t *testing.T) {
var m map[string]interface{} var m map[string]interface{}
err = decoder(input).Decode(&m) err = decoder(input).Decode(&m)
assert.NoError(t, err)
} }
func TestDecoderStrictValid(t *testing.T) { func TestDecoderStrictValid(t *testing.T) {
@@ -2053,6 +2062,19 @@ func (d *docUnmarshalTOML) UnmarshalTOML(i interface{}) error {
return nil return nil
} }
func TestDecoderStrictCustomUnmarshal(t *testing.T) {
t.Skip()
//input := `key = "ok"`
//var doc docUnmarshalTOML
//err := NewDecoder(bytes.NewReader([]byte(input))).Strict(true).Decode(&doc)
//if err != nil {
// t.Fatal("unexpected error:", err)
//}
//if doc.Decoded.Key != "ok" {
// t.Errorf("Bad unmarshal: expected ok, got %v", doc.Decoded.Key)
//}
}
type parent struct { type parent struct {
Doc docUnmarshalTOML Doc docUnmarshalTOML
DocPointer *docUnmarshalTOML DocPointer *docUnmarshalTOML
@@ -2256,7 +2278,7 @@ type Custom struct {
v string v string
} }
func (c *Custom) UnmarshalTOML(interface{}) error { func (c *Custom) UnmarshalTOML(v interface{}) error {
c.v = "called" c.v = "called"
return nil return nil
} }
@@ -2281,14 +2303,14 @@ type durationString struct {
time.Duration time.Duration
} }
func (d *durationString) UnmarshalTOML(interface{}) error { func (d *durationString) UnmarshalTOML(v interface{}) error {
d.Duration = 10 * time.Second d.Duration = 10 * time.Second
return nil return nil
} }
type config437Error struct{} type config437Error struct{}
func (e *config437Error) UnmarshalTOML(interface{}) error { func (e *config437Error) UnmarshalTOML(v interface{}) error {
return errors.New("expected") return errors.New("expected")
} }
+7 -8
View File
@@ -3,14 +3,13 @@ package testsuite
import ( import (
"fmt" "fmt"
"math" "math"
"strconv"
"time" "time"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
) )
// addTag adds JSON tags to a data structure as expected by toml-test. // addTag adds JSON tags to a data structure as expected by toml-test.
func addTag(tomlData interface{}) interface{} { func addTag(key string, tomlData interface{}) interface{} {
// Switch on the data type. // Switch on the data type.
switch orig := tomlData.(type) { switch orig := tomlData.(type) {
default: default:
@@ -22,7 +21,7 @@ func addTag(tomlData interface{}) interface{} {
case map[string]interface{}: case map[string]interface{}:
typed := make(map[string]interface{}, len(orig)) typed := make(map[string]interface{}, len(orig))
for k, v := range orig { for k, v := range orig {
typed[k] = addTag(v) typed[k] = addTag(k, v)
} }
return typed return typed
@@ -31,13 +30,13 @@ func addTag(tomlData interface{}) interface{} {
case []map[string]interface{}: case []map[string]interface{}:
typed := make([]map[string]interface{}, len(orig)) typed := make([]map[string]interface{}, len(orig))
for i, v := range orig { for i, v := range orig {
typed[i] = addTag(v).(map[string]interface{}) typed[i] = addTag("", v).(map[string]interface{})
} }
return typed return typed
case []interface{}: case []interface{}:
typed := make([]interface{}, len(orig)) typed := make([]interface{}, len(orig))
for i, v := range orig { for i, v := range orig {
typed[i] = addTag(v) typed[i] = addTag("", v)
} }
return typed return typed
@@ -53,11 +52,11 @@ func addTag(tomlData interface{}) interface{} {
// Tag primitive values: bool, string, int, and float64. // Tag primitive values: bool, string, int, and float64.
case bool: case bool:
return tag("bool", strconv.FormatBool(orig)) return tag("bool", fmt.Sprintf("%v", orig))
case string: case string:
return tag("string", orig) return tag("string", orig)
case int64: case int64:
return tag("integer", strconv.FormatInt(orig, 10)) return tag("integer", fmt.Sprintf("%d", orig))
case float64: case float64:
// Special case for nan since NaN == NaN is false. // Special case for nan since NaN == NaN is false.
if math.IsNaN(orig) { if math.IsNaN(orig) {
+10 -10
View File
@@ -9,7 +9,6 @@ import (
) )
func CmpJSON(t *testing.T, key string, want, have interface{}) { func CmpJSON(t *testing.T, key string, want, have interface{}) {
t.Helper()
switch w := want.(type) { switch w := want.(type) {
case map[string]interface{}: case map[string]interface{}:
cmpJSONMaps(t, key, w, have) cmpJSONMaps(t, key, w, have)
@@ -23,7 +22,6 @@ func CmpJSON(t *testing.T, key string, want, have interface{}) {
} }
func cmpJSONMaps(t *testing.T, key string, want map[string]interface{}, have interface{}) { func cmpJSONMaps(t *testing.T, key string, want map[string]interface{}, have interface{}) {
t.Helper()
haveMap, ok := have.(map[string]interface{}) haveMap, ok := have.(map[string]interface{})
if !ok { if !ok {
mismatch(t, key, "table", want, haveMap) mismatch(t, key, "table", want, haveMap)
@@ -63,7 +61,6 @@ func cmpJSONMaps(t *testing.T, key string, want map[string]interface{}, have int
} }
func cmpJSONArrays(t *testing.T, key string, want, have interface{}) { func cmpJSONArrays(t *testing.T, key string, want, have interface{}) {
t.Helper()
wantSlice, ok := want.([]interface{}) wantSlice, ok := want.([]interface{})
if !ok { if !ok {
panic(fmt.Sprintf("'value' should be a JSON array when 'type=array', but it is a %T", want)) panic(fmt.Sprintf("'value' should be a JSON array when 'type=array', but it is a %T", want))
@@ -86,7 +83,6 @@ func cmpJSONArrays(t *testing.T, key string, want, have interface{}) {
} }
func cmpJSONValues(t *testing.T, key string, want, have map[string]interface{}) { func cmpJSONValues(t *testing.T, key string, want, have map[string]interface{}) {
t.Helper()
wantType, ok := want["type"].(string) wantType, ok := want["type"].(string)
if !ok { if !ok {
panic(fmt.Sprintf("'type' should be a string, but it is a %T", want["type"])) panic(fmt.Sprintf("'type' should be a string, but it is a %T", want["type"]))
@@ -130,7 +126,6 @@ func cmpJSONValues(t *testing.T, key string, want, have map[string]interface{})
} }
func cmpAsStrings(t *testing.T, key string, want, have string) { func cmpAsStrings(t *testing.T, key string, want, have string) {
t.Helper()
if want != have { if want != have {
t.Fatalf("Values for key '%s' don't match:\n"+ t.Fatalf("Values for key '%s' don't match:\n"+
" Expected: %s\n"+ " Expected: %s\n"+
@@ -140,7 +135,6 @@ func cmpAsStrings(t *testing.T, key string, want, have string) {
} }
func cmpFloats(t *testing.T, key string, want, have string) { func cmpFloats(t *testing.T, key string, want, have string) {
t.Helper()
// Special case for NaN, since NaN != NaN. // Special case for NaN, since NaN != NaN.
if strings.HasSuffix(want, "nan") || strings.HasSuffix(have, "nan") { if strings.HasSuffix(want, "nan") || strings.HasSuffix(have, "nan") {
if want != have { if want != have {
@@ -183,7 +177,6 @@ var layouts = map[string]string{
} }
func cmpAsDatetimes(t *testing.T, key string, kind, want, have string) { func cmpAsDatetimes(t *testing.T, key string, kind, want, have string) {
t.Helper()
layout, ok := layouts[kind] layout, ok := layouts[kind]
if !ok { if !ok {
panic("should never happen") panic("should never happen")
@@ -207,6 +200,15 @@ func cmpAsDatetimes(t *testing.T, key string, kind, want, have string) {
} }
} }
func cmpAsDatetimesLocal(t *testing.T, key string, want, have string) {
if datetimeRepl.Replace(want) != datetimeRepl.Replace(have) {
t.Fatalf("Values for key '%s' don't match:\n"+
" Expected: %v\n"+
" Your encoder: %v",
key, want, have)
}
}
func kjoin(old, key string) string { func kjoin(old, key string) string {
if len(old) == 0 { if len(old) == 0 {
return key return key
@@ -228,7 +230,6 @@ func isValue(m map[string]interface{}) bool {
} }
func mismatch(t *testing.T, key string, wantType string, want, have interface{}) { func mismatch(t *testing.T, key string, wantType string, want, have interface{}) {
t.Helper()
t.Fatalf("Key '%s' is not an %s but %[4]T:\n"+ t.Fatalf("Key '%s' is not an %s but %[4]T:\n"+
" Expected: %#[3]v\n"+ " Expected: %#[3]v\n"+
" Your encoder: %#[4]v", " Your encoder: %#[4]v",
@@ -236,9 +237,8 @@ func mismatch(t *testing.T, key string, wantType string, want, have interface{})
} }
func valMismatch(t *testing.T, key string, wantType, haveType string, want, have interface{}) { func valMismatch(t *testing.T, key string, wantType, haveType string, want, have interface{}) {
t.Helper()
t.Fatalf("Key '%s' is not an %s but %s:\n"+ t.Fatalf("Key '%s' is not an %s but %s:\n"+
" Expected: %#[3]v\n"+ " Expected: %#[3]v\n"+
" Your encoder: %#[4]v", " Your encoder: %#[4]v",
key, wantType, haveType, want, have) key, wantType, want, have)
} }
+69
View File
@@ -0,0 +1,69 @@
package testsuite
import (
"bytes"
"encoding/json"
"fmt"
"github.com/pelletier/go-toml/v2"
)
type parser struct{}
func (p parser) Decode(input string) (output string, outputIsError bool, retErr error) {
defer func() {
if r := recover(); r != nil {
switch rr := r.(type) {
case error:
retErr = rr
default:
retErr = fmt.Errorf("%s", rr)
}
}
}()
var v interface{}
if err := toml.Unmarshal([]byte(input), &v); err != nil {
return err.Error(), true, nil
}
j, err := json.MarshalIndent(addTag("", v), "", " ")
if err != nil {
return "", false, retErr
}
return string(j), false, retErr
}
func (p parser) Encode(input string) (output string, outputIsError bool, retErr error) {
defer func() {
if r := recover(); r != nil {
switch rr := r.(type) {
case error:
retErr = rr
default:
retErr = fmt.Errorf("%s", rr)
}
}
}()
var tmp interface{}
err := json.Unmarshal([]byte(input), &tmp)
if err != nil {
return "", false, err
}
rm, err := rmTag(tmp)
if err != nil {
return err.Error(), true, retErr
}
buf := new(bytes.Buffer)
err = toml.NewEncoder(buf).Encode(rm)
if err != nil {
return err.Error(), true, retErr
}
return buf.String(), false, retErr
}
+17 -4
View File
@@ -5,11 +5,11 @@ import (
"strconv" "strconv"
"time" "time"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
) )
// Remove JSON tags to a data structure as returned by toml-test. // Remove JSON tags to a data structure as returned by toml-test.
func rmTag(typedJSON interface{}) (interface{}, error) { func rmTag(typedJson interface{}) (interface{}, error) {
// Check if key is in the table m. // Check if key is in the table m.
in := func(key string, m map[string]interface{}) bool { in := func(key string, m map[string]interface{}) bool {
_, ok := m[key] _, ok := m[key]
@@ -17,7 +17,8 @@ func rmTag(typedJSON interface{}) (interface{}, error) {
} }
// Switch on the data type. // Switch on the data type.
switch v := typedJSON.(type) { switch v := typedJson.(type) {
// Object: this can either be a TOML table or a primitive with tags. // Object: this can either be a TOML table or a primitive with tags.
case map[string]interface{}: case map[string]interface{}:
// This value represents a primitive: remove the tags and return just // This value represents a primitive: remove the tags and return just
@@ -55,7 +56,7 @@ func rmTag(typedJSON interface{}) (interface{}, error) {
} }
// The top level must be an object or array. // The top level must be an object or array.
return nil, fmt.Errorf("unrecognized JSON format '%T'", typedJSON) return nil, fmt.Errorf("unrecognized JSON format '%T'", typedJson)
} }
// Return a primitive: read the "type" and convert the "value" to that. // Return a primitive: read the "type" and convert the "value" to that.
@@ -114,3 +115,15 @@ func untag(typed map[string]interface{}) (interface{}, error) {
return nil, fmt.Errorf("untag: unrecognized tag type %q", t) return nil, fmt.Errorf("untag: unrecognized tag type %q", t)
} }
func parseTime(v, format string, local bool) (t time.Time, err error) {
if local {
t, err = time.ParseInLocation(format, v, time.Local)
} else {
t, err = time.Parse(format, v)
}
if err != nil {
return time.Time{}, fmt.Errorf("Could not parse %q as a datetime: %w", v, err)
}
return t, nil
}
+5 -5
View File
@@ -7,7 +7,7 @@ import (
"fmt" "fmt"
"os" "os"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
) )
// Marshal is a helper function for calling toml.Marshal // Marshal is a helper function for calling toml.Marshal
@@ -27,7 +27,7 @@ func Unmarshal(data []byte, v interface{}) error {
// ValueToTaggedJSON takes a data structure and returns the tagged JSON // ValueToTaggedJSON takes a data structure and returns the tagged JSON
// representation. // representation.
func ValueToTaggedJSON(doc interface{}) ([]byte, error) { func ValueToTaggedJSON(doc interface{}) ([]byte, error) {
return json.MarshalIndent(addTag(doc), "", " ") return json.MarshalIndent(addTag("", doc), "", " ")
} }
// DecodeStdin is a helper function for the toml-test binary interface. TOML input // DecodeStdin is a helper function for the toml-test binary interface. TOML input
@@ -37,13 +37,13 @@ func DecodeStdin() error {
var decoded map[string]interface{} var decoded map[string]interface{}
if err := toml.NewDecoder(os.Stdin).Decode(&decoded); err != nil { if err := toml.NewDecoder(os.Stdin).Decode(&decoded); err != nil {
return fmt.Errorf("error decoding TOML: %w", err) return fmt.Errorf("Error decoding TOML: %s", err)
} }
j := json.NewEncoder(os.Stdout) j := json.NewEncoder(os.Stdout)
j.SetIndent("", " ") j.SetIndent("", " ")
if err := j.Encode(addTag(decoded)); err != nil { if err := j.Encode(addTag("", decoded)); err != nil {
return fmt.Errorf("error encoding JSON: %w", err) return fmt.Errorf("Error encoding JSON: %s", err)
} }
return nil return nil
+2 -2
View File
@@ -1,6 +1,6 @@
package tracker package tracker
import "git.ostiwe.com/ostiwe/go-toml/v2/unstable" import "github.com/pelletier/go-toml/v2/unstable"
// KeyTracker is a tracker that keeps track of the current Key as the AST is // KeyTracker is a tracker that keeps track of the current Key as the AST is
// walked. // walked.
@@ -36,7 +36,7 @@ func (t *KeyTracker) Pop(node *unstable.Node) {
} }
} }
// Key returns the current key. // Key returns the current key
func (t *KeyTracker) Key() []string { func (t *KeyTracker) Key() []string {
k := make([]string, len(t.k)) k := make([]string, len(t.k))
copy(k, t.k) copy(k, t.k)
+7 -8
View File
@@ -5,7 +5,7 @@ import (
"fmt" "fmt"
"sync" "sync"
"git.ostiwe.com/ostiwe/go-toml/v2/unstable" "github.com/pelletier/go-toml/v2/unstable"
) )
type keyKind uint8 type keyKind uint8
@@ -288,12 +288,11 @@ func (s *SeenTracker) checkKeyValue(node *unstable.Node) (bool, error) {
idx = s.create(parentIdx, k, tableKind, false, true) idx = s.create(parentIdx, k, tableKind, false, true)
} else { } else {
entry := s.entries[idx] entry := s.entries[idx]
switch { if it.IsLast() {
case it.IsLast():
return false, fmt.Errorf("toml: key %s is already defined", string(k)) return false, fmt.Errorf("toml: key %s is already defined", string(k))
case entry.kind != tableKind: } else if entry.kind != tableKind {
return false, fmt.Errorf("toml: expected %s to be a table, not a %s", string(k), entry.kind) return false, fmt.Errorf("toml: expected %s to be a table, not a %s", string(k), entry.kind)
case entry.explicit: } else if entry.explicit {
return false, fmt.Errorf("toml: cannot redefine table %s that has already been explicitly defined", string(k)) return false, fmt.Errorf("toml: cannot redefine table %s that has already been explicitly defined", string(k))
} }
} }
@@ -310,16 +309,16 @@ func (s *SeenTracker) checkKeyValue(node *unstable.Node) (bool, error) {
return s.checkInlineTable(value) return s.checkInlineTable(value)
case unstable.Array: case unstable.Array:
return s.checkArray(value) return s.checkArray(value)
default:
return false, nil
} }
return false, nil
} }
func (s *SeenTracker) checkArray(node *unstable.Node) (first bool, err error) { func (s *SeenTracker) checkArray(node *unstable.Node) (first bool, err error) {
it := node.Children() it := node.Children()
for it.Next() { for it.Next() {
n := it.Node() n := it.Node()
switch n.Kind { //nolint:exhaustive switch n.Kind {
case unstable.InlineTable: case unstable.InlineTable:
first, err = s.checkInlineTable(n) first, err = s.checkInlineTable(n)
if err != nil { if err != nil {
+3 -4
View File
@@ -4,7 +4,7 @@ import (
"reflect" "reflect"
"testing" "testing"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
func TestEntrySize(t *testing.T) { func TestEntrySize(t *testing.T) {
@@ -12,10 +12,9 @@ func TestEntrySize(t *testing.T) {
// performance of unmarshaling documents. Should only be increased with care // performance of unmarshaling documents. Should only be increased with care
// and a very good reason. // and a very good reason.
maxExpectedEntrySize := 48 maxExpectedEntrySize := 48
entrySize := int(reflect.TypeOf(entry{}).Size())
assert.True(t, assert.True(t,
entrySize <= maxExpectedEntrySize, int(reflect.TypeOf(entry{}).Size()) <= maxExpectedEntrySize,
"Expected entry to be less than or equal to %d, got: %d", "Expected entry to be less than or equal to %d, got: %d",
maxExpectedEntrySize, entrySize, maxExpectedEntrySize, int(reflect.TypeOf(entry{}).Size()),
) )
} }
-1
View File
@@ -1,2 +1 @@
// Package tracker provides functions for keeping track of AST nodes.
package tracker package tracker
+2 -2
View File
@@ -5,7 +5,7 @@ import (
"strings" "strings"
"time" "time"
"git.ostiwe.com/ostiwe/go-toml/v2/unstable" "github.com/pelletier/go-toml/v2/unstable"
) )
// LocalDate represents a calendar day in no specific timezone. // LocalDate represents a calendar day in no specific timezone.
@@ -45,7 +45,7 @@ func (d *LocalDate) UnmarshalText(b []byte) error {
type LocalTime struct { type LocalTime struct {
Hour int // Hour of the day: [0; 24[ Hour int // Hour of the day: [0; 24[
Minute int // Minute of the hour: [0; 60[ Minute int // Minute of the hour: [0; 60[
Second int // Second of the minute: [0; 59] Second int // Second of the minute: [0; 60[
Nanosecond int // Nanoseconds within the second: [0, 1000000000[ Nanosecond int // Nanoseconds within the second: [0, 1000000000[
Precision int // Number of digits to display for Nanosecond. Precision int // Number of digits to display for Nanosecond.
} }
+2 -2
View File
@@ -4,8 +4,8 @@ import (
"testing" "testing"
"time" "time"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
func TestLocalDate_AsTime(t *testing.T) { func TestLocalDate_AsTime(t *testing.T) {
+51 -86
View File
@@ -4,7 +4,6 @@ import (
"bytes" "bytes"
"encoding" "encoding"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"io" "io"
"math" "math"
@@ -15,7 +14,7 @@ import (
"time" "time"
"unicode" "unicode"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/characters" "github.com/pelletier/go-toml/v2/internal/characters"
) )
// Marshal serializes a Go value as a TOML document. // Marshal serializes a Go value as a TOML document.
@@ -43,7 +42,7 @@ type Encoder struct {
arraysMultiline bool arraysMultiline bool
indentSymbol string indentSymbol string
indentTables bool indentTables bool
marshalJSONNumbers bool marshalJsonNumbers bool
} }
// NewEncoder returns a new Encoder that writes to w. // NewEncoder returns a new Encoder that writes to w.
@@ -90,14 +89,14 @@ func (enc *Encoder) SetIndentTables(indent bool) *Encoder {
return enc return enc
} }
// SetMarshalJSONNumbers forces the encoder to serialize `json.Number` as a // SetMarshalJsonNumbers forces the encoder to serialize `json.Number` as a
// float or integer instead of relying on TextMarshaler to emit a string. // float or integer instead of relying on TextMarshaler to emit a string.
// //
// *Unstable:* This method does not follow the compatibility guarantees of // *Unstable:* This method does not follow the compatibility guarantees of
// semver. It can be changed or removed without a new major version being // semver. It can be changed or removed without a new major version being
// issued. // issued.
func (enc *Encoder) SetMarshalJSONNumbers(indent bool) *Encoder { func (enc *Encoder) SetMarshalJsonNumbers(indent bool) *Encoder {
enc.marshalJSONNumbers = indent enc.marshalJsonNumbers = indent
return enc return enc
} }
@@ -180,7 +179,7 @@ func (enc *Encoder) Encode(v interface{}) error {
ctx.inline = enc.tablesInline ctx.inline = enc.tablesInline
if v == nil { if v == nil {
return errors.New("toml: cannot encode a nil interface") return fmt.Errorf("toml: cannot encode a nil interface")
} }
b, err := enc.encode(b, ctx, reflect.ValueOf(v)) b, err := enc.encode(b, ctx, reflect.ValueOf(v))
@@ -270,17 +269,18 @@ func (enc *Encoder) encode(b []byte, ctx encoderCtx, v reflect.Value) ([]byte, e
case LocalDateTime: case LocalDateTime:
return append(b, x.String()...), nil return append(b, x.String()...), nil
case json.Number: case json.Number:
if enc.marshalJSONNumbers { if enc.marshalJsonNumbers {
if x == "" { /// Useful zero value. if x == "" { /// Useful zero value.
return append(b, "0"...), nil return append(b, "0"...), nil
} else if v, err := x.Int64(); err == nil { } else if v, err := x.Int64(); err == nil {
return enc.encode(b, ctx, reflect.ValueOf(v)) return enc.encode(b, ctx, reflect.ValueOf(v))
} else if f, err := x.Float64(); err == nil { } else if f, err := x.Float64(); err == nil {
return enc.encode(b, ctx, reflect.ValueOf(f)) return enc.encode(b, ctx, reflect.ValueOf(f))
} } else {
return nil, fmt.Errorf("toml: unable to convert %q to int64 or float64", x) return nil, fmt.Errorf("toml: unable to convert %q to int64 or float64", x)
} }
} }
}
hasTextMarshaler := v.Type().Implements(textMarshalerType) hasTextMarshaler := v.Type().Implements(textMarshalerType)
if hasTextMarshaler || (v.CanAddr() && reflect.PointerTo(v.Type()).Implements(textMarshalerType)) { if hasTextMarshaler || (v.CanAddr() && reflect.PointerTo(v.Type()).Implements(textMarshalerType)) {
@@ -312,7 +312,7 @@ func (enc *Encoder) encode(b []byte, ctx encoderCtx, v reflect.Value) ([]byte, e
return enc.encodeSlice(b, ctx, v) return enc.encodeSlice(b, ctx, v)
case reflect.Interface: case reflect.Interface:
if v.IsNil() { if v.IsNil() {
return nil, errors.New("toml: encoding a nil interface is not supported") return nil, fmt.Errorf("toml: encoding a nil interface is not supported")
} }
return enc.encode(b, ctx, v.Elem()) return enc.encode(b, ctx, v.Elem())
@@ -329,30 +329,28 @@ func (enc *Encoder) encode(b []byte, ctx encoderCtx, v reflect.Value) ([]byte, e
case reflect.Float32: case reflect.Float32:
f := v.Float() f := v.Float()
switch { if math.IsNaN(f) {
case math.IsNaN(f):
b = append(b, "nan"...) b = append(b, "nan"...)
case f > math.MaxFloat32: } else if f > math.MaxFloat32 {
b = append(b, "inf"...) b = append(b, "inf"...)
case f < -math.MaxFloat32: } else if f < -math.MaxFloat32 {
b = append(b, "-inf"...) b = append(b, "-inf"...)
case math.Trunc(f) == f: } else if math.Trunc(f) == f {
b = strconv.AppendFloat(b, f, 'f', 1, 32) b = strconv.AppendFloat(b, f, 'f', 1, 32)
default: } else {
b = strconv.AppendFloat(b, f, 'f', -1, 32) b = strconv.AppendFloat(b, f, 'f', -1, 32)
} }
case reflect.Float64: case reflect.Float64:
f := v.Float() f := v.Float()
switch { if math.IsNaN(f) {
case math.IsNaN(f):
b = append(b, "nan"...) b = append(b, "nan"...)
case f > math.MaxFloat64: } else if f > math.MaxFloat64 {
b = append(b, "inf"...) b = append(b, "inf"...)
case f < -math.MaxFloat64: } else if f < -math.MaxFloat64 {
b = append(b, "-inf"...) b = append(b, "-inf"...)
case math.Trunc(f) == f: } else if math.Trunc(f) == f {
b = strconv.AppendFloat(b, f, 'f', 1, 64) b = strconv.AppendFloat(b, f, 'f', 1, 64)
default: } else {
b = strconv.AppendFloat(b, f, 'f', -1, 64) b = strconv.AppendFloat(b, f, 'f', -1, 64)
} }
case reflect.Bool: case reflect.Bool:
@@ -390,28 +388,7 @@ func shouldOmitEmpty(options valueOptions, v reflect.Value) bool {
} }
func shouldOmitZero(options valueOptions, v reflect.Value) bool { func shouldOmitZero(options valueOptions, v reflect.Value) bool {
if !options.omitzero { return options.omitzero && v.IsZero()
return false
}
// Check if the type implements isZeroer interface (has a custom IsZero method).
if v.Type().Implements(isZeroerType) {
return v.Interface().(isZeroer).IsZero()
}
// Check if pointer type implements isZeroer.
if reflect.PointerTo(v.Type()).Implements(isZeroerType) {
if v.CanAddr() {
return v.Addr().Interface().(isZeroer).IsZero()
}
// Create a temporary addressable copy to call the pointer receiver method.
pv := reflect.New(v.Type())
pv.Elem().Set(v)
return pv.Interface().(isZeroer).IsZero()
}
// Fall back to reflect's IsZero for types without custom IsZero method.
return v.IsZero()
} }
func (enc *Encoder) encodeKv(b []byte, ctx encoderCtx, options valueOptions, v reflect.Value) ([]byte, error) { func (enc *Encoder) encodeKv(b []byte, ctx encoderCtx, options valueOptions, v reflect.Value) ([]byte, error) {
@@ -464,9 +441,8 @@ func isEmptyValue(v reflect.Value) bool {
return v.Float() == 0 return v.Float() == 0
case reflect.Interface, reflect.Ptr: case reflect.Interface, reflect.Ptr:
return v.IsNil() return v.IsNil()
default:
return false
} }
return false
} }
func isEmptyStruct(v reflect.Value) bool { func isEmptyStruct(v reflect.Value) bool {
@@ -510,7 +486,7 @@ func (enc *Encoder) encodeString(b []byte, v string, options valueOptions) []byt
func needsQuoting(v string) bool { func needsQuoting(v string) bool {
// TODO: vectorize // TODO: vectorize
for _, b := range []byte(v) { for _, b := range []byte(v) {
if b == '\'' || b == '\r' || b == '\n' || characters.InvalidASCII(b) { if b == '\'' || b == '\r' || b == '\n' || characters.InvalidAscii(b) {
return true return true
} }
} }
@@ -604,9 +580,9 @@ func (enc *Encoder) encodeUnquotedKey(b []byte, v string) []byte {
return append(b, v...) return append(b, v...)
} }
func (enc *Encoder) encodeTableHeader(ctx encoderCtx, b []byte) []byte { func (enc *Encoder) encodeTableHeader(ctx encoderCtx, b []byte) ([]byte, error) {
if len(ctx.parentKey) == 0 { if len(ctx.parentKey) == 0 {
return b return b, nil
} }
b = enc.encodeComment(ctx.indent, ctx.options.comment, b) b = enc.encodeComment(ctx.indent, ctx.options.comment, b)
@@ -626,9 +602,10 @@ func (enc *Encoder) encodeTableHeader(ctx encoderCtx, b []byte) []byte {
b = append(b, "]\n"...) b = append(b, "]\n"...)
return b return b, nil
} }
//nolint:cyclop
func (enc *Encoder) encodeKey(b []byte, k string) []byte { func (enc *Encoder) encodeKey(b []byte, k string) []byte {
needsQuotation := false needsQuotation := false
cannotUseLiteral := false cannotUseLiteral := false
@@ -665,33 +642,30 @@ func (enc *Encoder) encodeKey(b []byte, k string) []byte {
func (enc *Encoder) keyToString(k reflect.Value) (string, error) { func (enc *Encoder) keyToString(k reflect.Value) (string, error) {
keyType := k.Type() keyType := k.Type()
if keyType.Implements(textMarshalerType) { switch {
case keyType.Kind() == reflect.String:
return k.String(), nil
case keyType.Implements(textMarshalerType):
keyB, err := k.Interface().(encoding.TextMarshaler).MarshalText() keyB, err := k.Interface().(encoding.TextMarshaler).MarshalText()
if err != nil { if err != nil {
return "", fmt.Errorf("toml: error marshalling key %v from text: %w", k, err) return "", fmt.Errorf("toml: error marshalling key %v from text: %w", k, err)
} }
return string(keyB), nil return string(keyB), nil
}
switch keyType.Kind() { case keyType.Kind() == reflect.Int || keyType.Kind() == reflect.Int8 || keyType.Kind() == reflect.Int16 || keyType.Kind() == reflect.Int32 || keyType.Kind() == reflect.Int64:
case reflect.String:
return k.String(), nil
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return strconv.FormatInt(k.Int(), 10), nil return strconv.FormatInt(k.Int(), 10), nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64: case keyType.Kind() == reflect.Uint || keyType.Kind() == reflect.Uint8 || keyType.Kind() == reflect.Uint16 || keyType.Kind() == reflect.Uint32 || keyType.Kind() == reflect.Uint64:
return strconv.FormatUint(k.Uint(), 10), nil return strconv.FormatUint(k.Uint(), 10), nil
case reflect.Float32: case keyType.Kind() == reflect.Float32:
return strconv.FormatFloat(k.Float(), 'f', -1, 32), nil return strconv.FormatFloat(k.Float(), 'f', -1, 32), nil
case reflect.Float64: case keyType.Kind() == reflect.Float64:
return strconv.FormatFloat(k.Float(), 'f', -1, 64), nil return strconv.FormatFloat(k.Float(), 'f', -1, 64), nil
default:
return "", fmt.Errorf("toml: type %s is not supported as a map key", keyType.Kind())
} }
return "", fmt.Errorf("toml: type %s is not supported as a map key", keyType.Kind())
} }
func (enc *Encoder) encodeMap(b []byte, ctx encoderCtx, v reflect.Value) ([]byte, error) { func (enc *Encoder) encodeMap(b []byte, ctx encoderCtx, v reflect.Value) ([]byte, error) {
@@ -704,19 +678,9 @@ func (enc *Encoder) encodeMap(b []byte, ctx encoderCtx, v reflect.Value) ([]byte
for iter.Next() { for iter.Next() {
v := iter.Value() v := iter.Value()
// Handle nil values: convert nil pointers to zero value, if isNil(v) {
// skip nil interfaces and nil maps.
switch v.Kind() {
case reflect.Ptr:
if v.IsNil() {
v = reflect.Zero(v.Type().Elem())
}
case reflect.Interface, reflect.Map:
if v.IsNil() {
continue continue
} }
default:
}
k, err := enc.keyToString(iter.Key()) k, err := enc.keyToString(iter.Key())
if err != nil { if err != nil {
@@ -805,9 +769,10 @@ func walkStruct(ctx encoderCtx, t *table, v reflect.Value) {
walkStruct(ctx, t, f.Elem()) walkStruct(ctx, t, f.Elem())
} }
continue continue
} } else {
k = fieldType.Name k = fieldType.Name
} }
}
if isNil(f) { if isNil(f) {
continue continue
@@ -926,7 +891,10 @@ func (enc *Encoder) encodeTable(b []byte, ctx encoderCtx, t table) ([]byte, erro
} }
if !ctx.skipTableHeader { if !ctx.skipTableHeader {
b = enc.encodeTableHeader(ctx, b) b, err = enc.encodeTableHeader(ctx, b)
if err != nil {
return nil, err
}
if enc.indentTables && len(ctx.parentKey) > 0 { if enc.indentTables && len(ctx.parentKey) > 0 {
ctx.indent++ ctx.indent++
@@ -939,7 +907,7 @@ func (enc *Encoder) encodeTable(b []byte, ctx encoderCtx, t table) ([]byte, erro
if shouldOmitEmpty(kv.Options, kv.Value) { if shouldOmitEmpty(kv.Options, kv.Value) {
continue continue
} }
if kv.Options.omitzero && shouldOmitZero(kv.Options, kv.Value) { if shouldOmitZero(kv.Options, kv.Value) {
continue continue
} }
hasNonEmptyKV = true hasNonEmptyKV = true
@@ -961,7 +929,7 @@ func (enc *Encoder) encodeTable(b []byte, ctx encoderCtx, t table) ([]byte, erro
if shouldOmitEmpty(table.Options, table.Value) { if shouldOmitEmpty(table.Options, table.Value) {
continue continue
} }
if table.Options.omitzero && shouldOmitZero(table.Options, table.Value) { if shouldOmitZero(table.Options, table.Value) {
continue continue
} }
if first { if first {
@@ -998,7 +966,7 @@ func (enc *Encoder) encodeTableInline(b []byte, ctx encoderCtx, t table) ([]byte
if shouldOmitEmpty(kv.Options, kv.Value) { if shouldOmitEmpty(kv.Options, kv.Value) {
continue continue
} }
if kv.Options.omitzero && shouldOmitZero(kv.Options, kv.Value) { if shouldOmitZero(kv.Options, kv.Value) {
continue continue
} }
@@ -1029,14 +997,11 @@ func willConvertToTable(ctx encoderCtx, v reflect.Value) bool {
if !v.IsValid() { if !v.IsValid() {
return false return false
} }
t := v.Type() if v.Type() == timeType || v.Type().Implements(textMarshalerType) || (v.Kind() != reflect.Ptr && v.CanAddr() && reflect.PointerTo(v.Type()).Implements(textMarshalerType)) {
if t == timeType || t.Implements(textMarshalerType) {
return false
}
if v.Kind() != reflect.Ptr && v.CanAddr() && reflect.PointerTo(t).Implements(textMarshalerType) {
return false return false
} }
t := v.Type()
switch t.Kind() { switch t.Kind() {
case reflect.Map, reflect.Struct: case reflect.Map, reflect.Struct:
return !ctx.inline return !ctx.inline
@@ -1110,7 +1075,7 @@ func (enc *Encoder) encodeSliceAsArrayTable(b []byte, ctx encoderCtx, v reflect.
scratch = enc.indent(ctx.indent, scratch) scratch = enc.indent(ctx.indent, scratch)
} }
scratch = append(scratch, "["...) scratch = append(scratch, "[["...)
for i, k := range ctx.parentKey { for i, k := range ctx.parentKey {
if i > 0 { if i > 0 {
@@ -1120,7 +1085,7 @@ func (enc *Encoder) encodeSliceAsArrayTable(b []byte, ctx encoderCtx, v reflect.
scratch = enc.encodeKey(scratch, k) scratch = enc.encodeKey(scratch, k)
} }
scratch = append(scratch, "]\n"...) scratch = append(scratch, "]]\n"...)
ctx.skipTableHeader = true ctx.skipTableHeader = true
b = enc.encodeComment(ctx.indent, ctx.options.comment, b) b = enc.encodeComment(ctx.indent, ctx.options.comment, b)
+16 -343
View File
@@ -3,7 +3,6 @@ package toml_test
import ( import (
"bytes" "bytes"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"math" "math"
"math/big" "math/big"
@@ -13,8 +12,8 @@ import (
"testing" "testing"
"time" "time"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
type marshalTextKey struct { type marshalTextKey struct {
@@ -29,7 +28,7 @@ func (k marshalTextKey) MarshalText() ([]byte, error) {
type marshalBadTextKey struct{} type marshalBadTextKey struct{}
func (k marshalBadTextKey) MarshalText() ([]byte, error) { func (k marshalBadTextKey) MarshalText() ([]byte, error) {
return nil, errors.New("error") return nil, fmt.Errorf("error")
} }
func toFloat(x interface{}) float64 { func toFloat(x interface{}) float64 {
@@ -45,7 +44,6 @@ func toFloat(x interface{}) float64 {
} }
func inDelta(t *testing.T, expected, actual interface{}, delta float64) { func inDelta(t *testing.T, expected, actual interface{}, delta float64) {
t.Helper()
dt := toFloat(expected) - toFloat(actual) dt := toFloat(expected) - toFloat(actual)
assert.True(t, assert.True(t,
dt < -delta && dt < delta, dt < -delta && dt < delta,
@@ -619,36 +617,12 @@ hello = 'world'
expected: ``, expected: ``,
}, },
{ {
desc: "nil interface value in map is ignored", desc: "nil value in map is ignored",
v: map[string]interface{}{ v: map[string]interface{}{
"A": nil, "A": nil,
}, },
expected: ``, expected: ``,
}, },
{
desc: "nil pointer to struct in map produces empty table",
v: map[string]*struct{}{
"A": nil,
},
expected: `[A]
`,
},
{
desc: "nil pointer to int in map produces zero value",
v: map[string]*int{
"A": nil,
},
expected: `A = 0
`,
},
{
desc: "nil pointer to string in map produces empty string",
v: map[string]*string{
"A": nil,
},
expected: `A = ''
`,
},
{ {
desc: "new line in table key", desc: "new line in table key",
v: map[string]interface{}{ v: map[string]interface{}{
@@ -968,6 +942,7 @@ nan = nan
assert.Equal(t, expected, string(actual)) assert.Equal(t, expected, string(actual))
} }
//nolint:funlen
func TestMarshalIndentTables(t *testing.T) { func TestMarshalIndentTables(t *testing.T) {
examples := []struct { examples := []struct {
desc string desc string
@@ -1036,7 +1011,7 @@ type customTextMarshaler struct {
func (c *customTextMarshaler) MarshalText() ([]byte, error) { func (c *customTextMarshaler) MarshalText() ([]byte, error) {
if c.value == 1 { if c.value == 1 {
return nil, errors.New("cannot represent 1 because this is a silly test") return nil, fmt.Errorf("cannot represent 1 because this is a silly test")
} }
return []byte(fmt.Sprintf("::%d", c.value)), nil return []byte(fmt.Sprintf("::%d", c.value)), nil
} }
@@ -1076,7 +1051,7 @@ func TestMarshalTextMarshaler(t *testing.T) {
type brokenWriter struct{} type brokenWriter struct{}
func (b *brokenWriter) Write([]byte) (int, error) { func (b *brokenWriter) Write([]byte) (int, error) {
return 0, errors.New("dead") return 0, fmt.Errorf("dead")
} }
func TestEncodeToBrokenWriter(t *testing.T) { func TestEncodeToBrokenWriter(t *testing.T) {
@@ -1099,10 +1074,10 @@ func TestEncoderSetIndentSymbol(t *testing.T) {
assert.Equal(t, expected, w.String()) assert.Equal(t, expected, w.String())
} }
func TestEncoderSetMarshalJSONNumbers(t *testing.T) { func TestEncoderSetMarshalJsonNumbers(t *testing.T) {
var w strings.Builder var w strings.Builder
enc := toml.NewEncoder(&w) enc := toml.NewEncoder(&w)
enc.SetMarshalJSONNumbers(true) enc.SetMarshalJsonNumbers(true)
err := enc.Encode(map[string]interface{}{ err := enc.Encode(map[string]interface{}{
"A": json.Number("1.1"), "A": json.Number("1.1"),
"B": json.Number("42e-3"), "B": json.Number("42e-3"),
@@ -1219,291 +1194,11 @@ IP = '192.168.178.35'
assert.Equal(t, expected, string(b)) assert.Equal(t, expected, string(b))
} }
// customZeroType has a custom IsZero method that returns true
// when Value is less than 10.
type customZeroType struct {
Value int
}
func (c customZeroType) IsZero() bool {
return c.Value < 10
}
// customZeroPointerType has a custom IsZero method on the pointer receiver.
type customZeroPointerType struct {
Value int
}
func (c *customZeroPointerType) IsZero() bool {
return c.Value < 10
}
func TestEncoderOmitzeroCustomIsZero(t *testing.T) {
type doc struct {
Custom customZeroType `toml:",omitzero"`
Normal int `toml:",omitzero"`
}
// Custom.Value = 5, which is < 10, so custom IsZero returns true
d := doc{
Custom: customZeroType{Value: 5},
Normal: 0,
}
b, err := toml.Marshal(d)
assert.NoError(t, err)
// Both fields should be omitted: Custom because custom IsZero returns true,
// Normal because its reflect zero value is true.
expected := ``
assert.Equal(t, expected, string(b))
}
func TestEncoderOmitzeroCustomIsZeroNotZero(t *testing.T) {
type doc struct {
Custom customZeroType `toml:",omitzero"`
Normal int `toml:",omitzero"`
}
// Custom.Value = 15, which is >= 10, so custom IsZero returns false
d := doc{
Custom: customZeroType{Value: 15},
Normal: 42,
}
b, err := toml.Marshal(d)
assert.NoError(t, err)
// Both fields should be present
expected := `Normal = 42
[Custom]
Value = 15
`
assert.Equal(t, expected, string(b))
}
func TestEncoderOmitzeroCustomIsZeroPointerReceiver(t *testing.T) {
type doc struct {
Custom customZeroPointerType `toml:",omitzero"`
}
// Custom.Value = 5, which is < 10, so custom IsZero returns true
d := doc{
Custom: customZeroPointerType{Value: 5},
}
b, err := toml.Marshal(d)
assert.NoError(t, err)
// Field should be omitted because custom IsZero returns true
expected := ``
assert.Equal(t, expected, string(b))
}
func TestEncoderOmitzeroCustomIsZeroPointerReceiverNotZero(t *testing.T) {
type doc struct {
Custom customZeroPointerType `toml:",omitzero"`
}
// Custom.Value = 15, which is >= 10, so custom IsZero returns false
d := doc{
Custom: customZeroPointerType{Value: 15},
}
b, err := toml.Marshal(d)
assert.NoError(t, err)
// Field should be present
expected := `[Custom]
Value = 15
`
assert.Equal(t, expected, string(b))
}
// TestEncoderOmitzeroCustomIsZeroPointerReceiverAddressable tests the v.CanAddr() path
// by marshaling a pointer to a struct, which makes fields addressable.
func TestEncoderOmitzeroCustomIsZeroPointerReceiverAddressable(t *testing.T) {
type doc struct {
Custom customZeroPointerType `toml:",omitzero"`
}
// Custom.Value = 5, which is < 10, so custom IsZero returns true
d := &doc{
Custom: customZeroPointerType{Value: 5},
}
b, err := toml.Marshal(d)
assert.NoError(t, err)
// Field should be omitted because custom IsZero returns true
expected := ``
assert.Equal(t, expected, string(b))
}
// TestEncoderOmitzeroCustomIsZeroPointerReceiverAddressableNotZero tests the v.CanAddr() path
// when custom IsZero returns false.
func TestEncoderOmitzeroCustomIsZeroPointerReceiverAddressableNotZero(t *testing.T) {
type doc struct {
Custom customZeroPointerType `toml:",omitzero"`
}
// Custom.Value = 15, which is >= 10, so custom IsZero returns false
d := &doc{
Custom: customZeroPointerType{Value: 15},
}
b, err := toml.Marshal(d)
assert.NoError(t, err)
// Field should be present
expected := `[Custom]
Value = 15
`
assert.Equal(t, expected, string(b))
}
// TestEncoderOmitzeroCustomIsZeroInlineTable tests omitzero with inline tables.
func TestEncoderOmitzeroCustomIsZeroInlineTable(t *testing.T) {
type doc struct {
Custom customZeroType `toml:",omitzero,inline"`
}
// Custom.Value = 5, which is < 10, so custom IsZero returns true
d := doc{
Custom: customZeroType{Value: 5},
}
b, err := toml.Marshal(d)
assert.NoError(t, err)
// Field should be omitted
expected := ``
assert.Equal(t, expected, string(b))
}
// TestEncoderOmitzeroCustomIsZeroInlineTableNotZero tests omitzero with inline tables when not zero.
func TestEncoderOmitzeroCustomIsZeroInlineTableNotZero(t *testing.T) {
type doc struct {
Custom customZeroType `toml:",omitzero,inline"`
}
// Custom.Value = 15, which is >= 10, so custom IsZero returns false
d := doc{
Custom: customZeroType{Value: 15},
}
b, err := toml.Marshal(d)
assert.NoError(t, err)
// Field should be present as inline table
expected := `Custom = {Value = 15}
`
assert.Equal(t, expected, string(b))
}
// TestEncoderOmitzeroCustomIsZeroMixedTypes tests omitzero with a mix of custom and regular types.
func TestEncoderOmitzeroCustomIsZeroMixedTypes(t *testing.T) {
type doc struct {
Custom customZeroType `toml:",omitzero"`
Regular int `toml:",omitzero"`
NoOmit customZeroType `toml:""`
Pointer *int `toml:",omitzero"`
}
d := doc{
Custom: customZeroType{Value: 5}, // IsZero returns true
Regular: 0, // zero value
NoOmit: customZeroType{Value: 5}, // not omitted (no omitzero tag)
Pointer: nil, // nil pointer
}
b, err := toml.Marshal(d)
assert.NoError(t, err)
// Custom is omitted (custom IsZero true), Regular is omitted (zero value),
// NoOmit is present (no omitzero tag), Pointer is omitted (nil)
expected := `[NoOmit]
Value = 5
`
assert.Equal(t, expected, string(b))
}
// TestEncoderOmitzeroCustomIsZeroSlice tests omitzero with slices containing custom types.
func TestEncoderOmitzeroCustomIsZeroSlice(t *testing.T) {
type doc struct {
Items []customZeroType `toml:",omitzero"`
}
// Nil slice should be omitted (IsZero returns true for nil slices)
d := doc{
Items: nil,
}
b, err := toml.Marshal(d)
assert.NoError(t, err)
expected := ``
assert.Equal(t, expected, string(b))
// Empty but non-nil slice is NOT zero, so it's included
d2 := doc{
Items: []customZeroType{},
}
b2, err := toml.Marshal(d2)
assert.NoError(t, err)
expected2 := `Items = []
`
assert.Equal(t, expected2, string(b2))
}
// TestEncoderOmitzeroCustomIsZeroNestedStruct tests omitzero with nested structs.
func TestEncoderOmitzeroCustomIsZeroNestedStruct(t *testing.T) {
type inner struct {
Custom customZeroType `toml:",omitzero"`
Value int `toml:",omitzero"`
}
type doc struct {
Inner inner `toml:",omitzero"`
}
// Inner struct has all zero fields, but the struct itself is not zero
// (reflect.Value.IsZero checks if all fields are zero)
d := doc{
Inner: inner{
Custom: customZeroType{Value: 5}, // custom IsZero returns true
Value: 0, // zero value
},
}
b, err := toml.Marshal(d)
assert.NoError(t, err)
// Inner is present but its fields are omitted
expected := `[Inner]
`
assert.Equal(t, expected, string(b))
}
func TestEncoderTagFieldName(t *testing.T) { func TestEncoderTagFieldName(t *testing.T) {
type doc struct { type doc struct {
String string `toml:"hello"` String string `toml:"hello"`
OkSym string `toml:"#"` OkSym string `toml:"#"`
Bad string `toml:"\"` //nolint:govet Bad string `toml:"\"`
} }
d := doc{String: "world"} d := doc{String: "world"}
@@ -2067,14 +1762,14 @@ func ExampleMarshal() {
func ExampleMarshal_commented() { func ExampleMarshal_commented() {
type Common struct { type Common struct {
Listen string `toml:"listen" comment:"general listener"` Listen string `toml:"listen" comment:"general listener"`
PprofListen string `toml:"pprof-listen" comment:"listener to serve /debug/pprof requests. '-pprof' argument overrides it"` //nolint:lll PprofListen string `toml:"pprof-listen" comment:"listener to serve /debug/pprof requests. '-pprof' argument overrides it"`
MaxMetricsPerTarget int `toml:"max-metrics-per-target" comment:"limit numbers of queried metrics per target in /render requests, 0 or negative = unlimited"` //nolint:lll MaxMetricsPerTarget int `toml:"max-metrics-per-target" comment:"limit numbers of queried metrics per target in /render requests, 0 or negative = unlimited"`
MemoryReturnInterval time.Duration `toml:"memory-return-interval" comment:"daemon will return the freed memory to the OS when it>0"` MemoryReturnInterval time.Duration `toml:"memory-return-interval" comment:"daemon will return the freed memory to the OS when it>0"`
} }
type Costs struct { type Costs struct {
Cost *int `toml:"cost" comment:"default cost (for wildcarded equivalence or matched with regex, or if no value cost set)"` Cost *int `toml:"cost" comment:"default cost (for wildcarded equivalence or matched with regex, or if no value cost set)"`
ValuesCost map[string]int `toml:"values-cost" comment:"cost with some value (for equivalence without wildcards) (additional tuning, usually not needed)"` //nolint:lll ValuesCost map[string]int `toml:"values-cost" comment:"cost with some value (for equivalence without wildcards) (additional tuning, usually not needed)"`
} }
type ClickHouse struct { type ClickHouse struct {
@@ -2089,7 +1784,7 @@ func ExampleMarshal_commented() {
DateTreeTableVersion int `toml:"date-tree-table-version,commented"` DateTreeTableVersion int `toml:"date-tree-table-version,commented"`
TreeTimeout time.Duration `toml:"tree-timeout,commented"` TreeTimeout time.Duration `toml:"tree-timeout,commented"`
TagTable string `toml:"tag-table,commented"` TagTable string `toml:"tag-table,commented"`
ExtraPrefix string `toml:"extra-prefix" comment:"add extra prefix (directory in graphite) for all metrics, w/o trailing dot"` //nolint:lll ExtraPrefix string `toml:"extra-prefix" comment:"add extra prefix (directory in graphite) for all metrics, w/o trailing dot"`
ConnectTimeout time.Duration `toml:"connect-timeout" comment:"TCP connection timeout"` ConnectTimeout time.Duration `toml:"connect-timeout" comment:"TCP connection timeout"`
DataTableLegacy string `toml:"data-table,commented"` DataTableLegacy string `toml:"data-table,commented"`
RollupConfLegacy string `toml:"rollup-conf,commented"` RollupConfLegacy string `toml:"rollup-conf,commented"`
@@ -2192,12 +1887,12 @@ func TestReadmeComments(t *testing.T) {
type Config struct { type Config struct {
Host string `toml:"host" comment:"Host IP to connect to."` Host string `toml:"host" comment:"Host IP to connect to."`
Port int `toml:"port" comment:"Port of the remote server."` Port int `toml:"port" comment:"Port of the remote server."`
TLS TLS `toml:"TLS,commented" comment:"Encryption parameters (optional)"` Tls TLS `toml:"TLS,commented" comment:"Encryption parameters (optional)"`
} }
example := Config{ example := Config{
Host: "127.0.0.1", Host: "127.0.0.1",
Port: 4242, Port: 4242,
TLS: TLS{ Tls: TLS{
Cipher: "AEAD-AES128-GCM-SHA256", Cipher: "AEAD-AES128-GCM-SHA256",
Version: "TLS 1.3", Version: "TLS 1.3",
}, },
@@ -2217,25 +1912,3 @@ port = 4242
` `
assert.Equal(t, expected, string(out)) assert.Equal(t, expected, string(out))
} }
// TestMarshalIssue975 tests that nil pointer values in maps are marshaled as
// empty tables, allowing round-trip marshaling to work correctly.
// See https://git.ostiwe.com/ostiwe/go-toml/issues/975
func TestMarshalIssue975(t *testing.T) {
// Test case from the issue: map[string]*struct{}
oldMap := map[string]*struct{}{
"foo": nil,
}
doc, err := toml.Marshal(&oldMap)
assert.NoError(t, err)
assert.Equal(t, "[foo]\n", string(doc))
var newMap map[string]*struct{}
err = toml.Unmarshal(doc, &newMap)
assert.NoError(t, err)
// Verify the key is preserved after round-trip
_, exists := newMap["foo"]
assert.True(t, exists, "key 'foo' should exist after round-trip")
}
+1 -3
View File
@@ -1,4 +1,3 @@
// Package ossfuzz provides a fuzzing target for OSS-Fuzz.
package ossfuzz package ossfuzz
import ( import (
@@ -6,10 +5,9 @@ import (
"reflect" "reflect"
"strings" "strings"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
) )
// FuzzToml is the fuzzing target.
func FuzzToml(data []byte) int { func FuzzToml(data []byte) int {
if len(data) >= 2048 { if len(data) >= 2048 {
return 0 return 0
+9 -17
View File
@@ -1,8 +1,8 @@
package toml package toml
import ( import (
"git.ostiwe.com/ostiwe/go-toml/v2/internal/tracker" "github.com/pelletier/go-toml/v2/internal/tracker"
"git.ostiwe.com/ostiwe/go-toml/v2/unstable" "github.com/pelletier/go-toml/v2/unstable"
) )
type strict struct { type strict struct {
@@ -12,9 +12,6 @@ type strict struct {
key tracker.KeyTracker key tracker.KeyTracker
missing []unstable.ParserError missing []unstable.ParserError
// Reference to the document for computing key ranges.
doc []byte
} }
func (s *strict) EnterTable(node *unstable.Node) { func (s *strict) EnterTable(node *unstable.Node) {
@@ -55,7 +52,7 @@ func (s *strict) MissingTable(node *unstable.Node) {
} }
s.missing = append(s.missing, unstable.ParserError{ s.missing = append(s.missing, unstable.ParserError{
Highlight: s.keyLocation(node), Highlight: keyLocation(node),
Message: "missing table", Message: "missing table",
Key: s.key.Key(), Key: s.key.Key(),
}) })
@@ -67,7 +64,7 @@ func (s *strict) MissingField(node *unstable.Node) {
} }
s.missing = append(s.missing, unstable.ParserError{ s.missing = append(s.missing, unstable.ParserError{
Highlight: s.keyLocation(node), Highlight: keyLocation(node),
Message: "missing field", Message: "missing field",
Key: s.key.Key(), Key: s.key.Key(),
}) })
@@ -90,7 +87,7 @@ func (s *strict) Error(doc []byte) error {
return err return err
} }
func (s *strict) keyLocation(node *unstable.Node) []byte { func keyLocation(node *unstable.Node) []byte {
k := node.Key() k := node.Key()
hasOne := k.Next() hasOne := k.Next()
@@ -98,17 +95,12 @@ func (s *strict) keyLocation(node *unstable.Node) []byte {
panic("should not be called with empty key") panic("should not be called with empty key")
} }
// Get the range from the first key to the last key. start := k.Node().Data
firstRaw := k.Node().Raw end := k.Node().Data
lastRaw := firstRaw
for k.Next() { for k.Next() {
lastRaw = k.Node().Raw end = k.Node().Data
} }
// Compute the slice from the document using the ranges. return start[:cap(start)-cap(end)+len(end)]
start := firstRaw.Offset
end := lastRaw.Offset + lastRaw.Length
return s.doc[start:end]
} }
+4 -5
View File
@@ -9,7 +9,7 @@ YELLOW='\033[1;33m'
BLUE='\033[0;34m' BLUE='\033[0;34m'
NC='\033[0m' # No Color NC='\033[0m' # No Color
# Go versions to test (1.11 through 1.26) # Go versions to test (1.11 through 1.25)
GO_VERSIONS=( GO_VERSIONS=(
"1.11" "1.11"
"1.12" "1.12"
@@ -26,7 +26,6 @@ GO_VERSIONS=(
"1.23" "1.23"
"1.24" "1.24"
"1.25" "1.25"
"1.26"
) )
# Default values # Default values
@@ -65,7 +64,7 @@ EXAMPLES:
$0 # Test all Go versions in parallel $0 # Test all Go versions in parallel
$0 --sequential # Test all Go versions sequentially $0 --sequential # Test all Go versions sequentially
$0 1.21 1.22 1.23 # Test specific versions $0 1.21 1.22 1.23 # Test specific versions
$0 --verbose --output ./results 1.25 1.26 # Verbose output to custom directory $0 --verbose --output ./results 1.24 1.25 # Verbose output to custom directory
EXIT CODES: EXIT CODES:
0 Recent Go versions pass (good compatibility) 0 Recent Go versions pass (good compatibility)
@@ -137,8 +136,8 @@ fi
# Validate Go versions # Validate Go versions
for version in "${GO_VERSIONS[@]}"; do for version in "${GO_VERSIONS[@]}"; do
if ! [[ "$version" =~ ^1\.(1[1-9]|2[0-6])$ ]]; then if ! [[ "$version" =~ ^1\.(1[1-9]|2[0-5])$ ]]; then
log_error "Invalid Go version: $version. Supported versions: 1.11-1.26" log_error "Invalid Go version: $version. Supported versions: 1.11-1.25"
exit 1 exit 1
fi fi
done done
+5 -6
View File
@@ -1,16 +1,16 @@
//go:generate go run github.com/toml-lang/toml-test/cmd/toml-test@v1.6.0 -copy ./tests //go:generate go run github.com/toml-lang/toml-test/cmd/toml-test@v1.6.0 -copy ./tests
//go:generate go run ./cmd/tomltestgen/main.go -r v1.6.0 -o toml_testgen_test.go //go:generate go run ./cmd/tomltestgen/main.go -r v1.6.0 -o toml_testgen_test.go
// This is a support file for toml_testgen_test.go
package toml_test package toml_test
import ( import (
"encoding/json" "encoding/json"
"errors"
"testing" "testing"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/testsuite" "github.com/pelletier/go-toml/v2/internal/testsuite"
) )
func testgenInvalid(t *testing.T, input string) { func testgenInvalid(t *testing.T, input string) {
@@ -39,8 +39,7 @@ func testgenValid(t *testing.T, input string, jsonRef string) {
err := testsuite.Unmarshal([]byte(input), &doc) err := testsuite.Unmarshal([]byte(input), &doc)
if err != nil { if err != nil {
de := &toml.DecodeError{} if de, ok := err.(*toml.DecodeError); ok {
if errors.As(err, &de) {
t.Logf("%s\n%s", err, de) t.Logf("%s\n%s", err, de)
} }
t.Fatalf("failed parsing toml: %s", err) t.Fatalf("failed parsing toml: %s", err)
+1 -1
View File
@@ -1,4 +1,4 @@
// Code generated by tomltestgen for toml-test ref v1.6.0 on 2025-10-22T16:33:06+11:00. DO NOT EDIT. // Generated by tomltestgen for toml-test ref v1.6.0 on 2025-10-22T16:33:06+11:00
package toml_test package toml_test
import ( import (
+6 -15
View File
@@ -6,18 +6,9 @@ import (
"time" "time"
) )
// isZeroer is used to check if a type has a custom IsZero method. var timeType = reflect.TypeOf((*time.Time)(nil)).Elem()
// This allows custom types to define their own zero-value semantics. var textMarshalerType = reflect.TypeOf((*encoding.TextMarshaler)(nil)).Elem()
type isZeroer interface { var textUnmarshalerType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
IsZero() bool var mapStringInterfaceType = reflect.TypeOf(map[string]interface{}(nil))
} var sliceInterfaceType = reflect.TypeOf([]interface{}(nil))
var stringType = reflect.TypeOf("")
var (
timeType = reflect.TypeOf((*time.Time)(nil)).Elem()
textMarshalerType = reflect.TypeOf((*encoding.TextMarshaler)(nil)).Elem()
textUnmarshalerType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
isZeroerType = reflect.TypeOf((*isZeroer)(nil)).Elem()
mapStringInterfaceType = reflect.TypeOf(map[string]interface{}(nil))
sliceInterfaceType = reflect.TypeOf([]interface{}(nil))
stringType = reflect.TypeOf("")
)
+49 -130
View File
@@ -12,8 +12,8 @@ import (
"sync/atomic" "sync/atomic"
"time" "time"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/tracker" "github.com/pelletier/go-toml/v2/internal/tracker"
"git.ostiwe.com/ostiwe/go-toml/v2/unstable" "github.com/pelletier/go-toml/v2/unstable"
) )
// Unmarshal deserializes a TOML document into a Go value. // Unmarshal deserializes a TOML document into a Go value.
@@ -56,18 +56,13 @@ func (d *Decoder) DisallowUnknownFields() *Decoder {
// EnableUnmarshalerInterface allows to enable unmarshaler interface. // EnableUnmarshalerInterface allows to enable unmarshaler interface.
// //
// With this feature enabled, types implementing the unstable.Unmarshaler // With this feature enabled, types implementing the unstable/Unmarshaler
// interface can be decoded from any structure of the document. It allows types // interface can be decoded from any structure of the document. It allows types
// that don't have a straightforward TOML representation to provide their own // that don't have a straightforward TOML representation to provide their own
// decoding logic. // decoding logic.
// //
// The UnmarshalTOML method receives raw TOML bytes: // Currently, types can only decode from a single value. Tables and array tables
// - For single values: the raw value bytes (e.g., `"hello"` for a string) // are not supported.
// - For tables: all key-value lines belonging to that table
// - For inline tables/arrays: the raw bytes of the inline structure
//
// The unstable.RawMessage type can be used to capture raw TOML bytes for
// later processing, similar to json.RawMessage.
// //
// *Unstable:* This method does not follow the compatibility guarantees of // *Unstable:* This method does not follow the compatibility guarantees of
// semver. It can be changed or removed without a new major version being // semver. It can be changed or removed without a new major version being
@@ -127,7 +122,6 @@ func (d *Decoder) Decode(v interface{}) error {
dec := decoder{ dec := decoder{
strict: strict{ strict: strict{
Enabled: d.strict, Enabled: d.strict,
doc: b,
}, },
unmarshalerInterface: d.unmarshalerInterface, unmarshalerInterface: d.unmarshalerInterface,
} }
@@ -231,7 +225,7 @@ func (d *decoder) FromParser(v interface{}) error {
} }
if r.IsNil() { if r.IsNil() {
return errors.New("toml: decoding pointer target cannot be nil") return fmt.Errorf("toml: decoding pointer target cannot be nil")
} }
r = r.Elem() r = r.Elem()
@@ -278,7 +272,7 @@ func (d *decoder) handleRootExpression(expr *unstable.Node, v reflect.Value) err
var err error var err error
var first bool // used for to clear array tables on first use var first bool // used for to clear array tables on first use
if !d.skipUntilTable || expr.Kind != unstable.KeyValue { if !(d.skipUntilTable && expr.Kind == unstable.KeyValue) {
first, err = d.seen.CheckExpression(expr) first, err = d.seen.CheckExpression(expr)
if err != nil { if err != nil {
return err return err
@@ -383,7 +377,7 @@ func (d *decoder) handleArrayTableCollectionLast(key unstable.Iterator, v reflec
case reflect.Array: case reflect.Array:
idx := d.arrayIndex(true, v) idx := d.arrayIndex(true, v)
if idx >= v.Len() { if idx >= v.Len() {
return v, fmt.Errorf("%w at position %d", d.typeMismatchError("array table", v.Type()), idx) return v, fmt.Errorf("%s at position %d", d.typeMismatchError("array table", v.Type()), idx)
} }
elem := v.Index(idx) elem := v.Index(idx)
_, err := d.handleArrayTable(key, elem) _, err := d.handleArrayTable(key, elem)
@@ -458,14 +452,14 @@ func (d *decoder) handleArrayTableCollection(key unstable.Iterator, v reflect.Va
case reflect.Array: case reflect.Array:
idx := d.arrayIndex(false, v) idx := d.arrayIndex(false, v)
if idx >= v.Len() { if idx >= v.Len() {
return v, fmt.Errorf("%w at position %d", d.typeMismatchError("array table", v.Type()), idx) return v, fmt.Errorf("%s at position %d", d.typeMismatchError("array table", v.Type()), idx)
} }
elem := v.Index(idx) elem := v.Index(idx)
_, err := d.handleArrayTable(key, elem) _, err := d.handleArrayTable(key, elem)
return v, err return v, err
default:
return d.handleArrayTable(key, v)
} }
return d.handleArrayTable(key, v)
} }
func (d *decoder) handleKeyPart(key unstable.Iterator, v reflect.Value, nextFn handlerFn, makeFn valueMakerFn) (reflect.Value, error) { func (d *decoder) handleKeyPart(key unstable.Iterator, v reflect.Value, nextFn handlerFn, makeFn valueMakerFn) (reflect.Value, error) {
@@ -499,8 +493,7 @@ func (d *decoder) handleKeyPart(key unstable.Iterator, v reflect.Value, nextFn h
mv := v.MapIndex(mk) mv := v.MapIndex(mk)
set := false set := false
switch { if !mv.IsValid() {
case !mv.IsValid():
// If there is no value in the map, create a new one according to // If there is no value in the map, create a new one according to
// the map type. If the element type is interface, create either a // the map type. If the element type is interface, create either a
// map[string]interface{} or a []interface{} depending on whether // map[string]interface{} or a []interface{} depending on whether
@@ -513,13 +506,13 @@ func (d *decoder) handleKeyPart(key unstable.Iterator, v reflect.Value, nextFn h
mv = reflect.New(t).Elem() mv = reflect.New(t).Elem()
} }
set = true set = true
case mv.Kind() == reflect.Interface: } else if mv.Kind() == reflect.Interface {
mv = mv.Elem() mv = mv.Elem()
if !mv.IsValid() { if !mv.IsValid() {
mv = makeFn() mv = makeFn()
} }
set = true set = true
case !mv.CanAddr(): } else if !mv.CanAddr() {
vt := v.Type() vt := v.Type()
t := vt.Elem() t := vt.Elem()
oldmv := mv oldmv := mv
@@ -604,8 +597,9 @@ func (d *decoder) handleArrayTablePart(key unstable.Iterator, v reflect.Value) (
// cannot handle it. // cannot handle it.
func (d *decoder) handleTable(key unstable.Iterator, v reflect.Value) (reflect.Value, error) { func (d *decoder) handleTable(key unstable.Iterator, v reflect.Value) (reflect.Value, error) {
if v.Kind() == reflect.Slice { if v.Kind() == reflect.Slice {
// For non-empty slices, work with the last element if v.Len() == 0 {
if v.Len() > 0 { return reflect.Value{}, unstable.NewParserError(key.Node().Data, "cannot store a table in a slice")
}
elem := v.Index(v.Len() - 1) elem := v.Index(v.Len() - 1)
x, err := d.handleTable(key, elem) x, err := d.handleTable(key, elem)
if err != nil { if err != nil {
@@ -616,17 +610,6 @@ func (d *decoder) handleTable(key unstable.Iterator, v reflect.Value) (reflect.V
} }
return reflect.Value{}, nil return reflect.Value{}, nil
} }
// Empty slice - check if it implements Unmarshaler (e.g., RawMessage)
// and we're at the end of the key path
if d.unmarshalerInterface && !key.Next() {
if v.CanAddr() && v.Addr().CanInterface() {
if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok {
return d.handleKeyValuesUnmarshaler(outi)
}
}
}
return reflect.Value{}, unstable.NewParserError(key.Node().Data, "cannot store a table in a slice")
}
if key.Next() { if key.Next() {
// Still scoping the key // Still scoping the key
return d.handleTablePart(key, v) return d.handleTablePart(key, v)
@@ -639,24 +622,6 @@ func (d *decoder) handleTable(key unstable.Iterator, v reflect.Value) (reflect.V
// Handle root expressions until the end of the document or the next // Handle root expressions until the end of the document or the next
// non-key-value. // non-key-value.
func (d *decoder) handleKeyValues(v reflect.Value) (reflect.Value, error) { func (d *decoder) handleKeyValues(v reflect.Value) (reflect.Value, error) {
// Check if target implements Unmarshaler before processing key-values.
// This allows types to handle entire tables themselves.
if d.unmarshalerInterface {
vv := v
for vv.Kind() == reflect.Ptr {
if vv.IsNil() {
vv.Set(reflect.New(vv.Type().Elem()))
}
vv = vv.Elem()
}
if vv.CanAddr() && vv.Addr().CanInterface() {
if outi, ok := vv.Addr().Interface().(unstable.Unmarshaler); ok {
// Collect all key-value expressions for this table
return d.handleKeyValuesUnmarshaler(outi)
}
}
}
var rv reflect.Value var rv reflect.Value
for d.nextExpr() { for d.nextExpr() {
expr := d.expr() expr := d.expr()
@@ -686,41 +651,6 @@ func (d *decoder) handleKeyValues(v reflect.Value) (reflect.Value, error) {
return rv, nil return rv, nil
} }
// handleKeyValuesUnmarshaler collects all key-value expressions for a table
// and passes them to the Unmarshaler as raw TOML bytes.
func (d *decoder) handleKeyValuesUnmarshaler(u unstable.Unmarshaler) (reflect.Value, error) {
// Collect raw bytes from all key-value expressions for this table.
// We use the Raw field on each KeyValue expression to preserve the
// original formatting (whitespace, quoting style, etc.) from the document.
var buf []byte
for d.nextExpr() {
expr := d.expr()
if expr.Kind != unstable.KeyValue {
d.stashExpr()
break
}
_, err := d.seen.CheckExpression(expr)
if err != nil {
return reflect.Value{}, err
}
// Use the raw bytes from the original document to preserve formatting
if expr.Raw.Length > 0 {
raw := d.p.Raw(expr.Raw)
buf = append(buf, raw...)
}
buf = append(buf, '\n')
}
if err := u.UnmarshalTOML(buf); err != nil {
return reflect.Value{}, err
}
return reflect.Value{}, nil
}
type ( type (
handlerFn func(key unstable.Iterator, v reflect.Value) (reflect.Value, error) handlerFn func(key unstable.Iterator, v reflect.Value) (reflect.Value, error)
valueMakerFn func() reflect.Value valueMakerFn func() reflect.Value
@@ -765,22 +695,15 @@ func (d *decoder) handleValue(value *unstable.Node, v reflect.Value) error {
if d.unmarshalerInterface { if d.unmarshalerInterface {
if v.CanAddr() && v.Addr().CanInterface() { if v.CanAddr() && v.Addr().CanInterface() {
if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok { if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok {
// Pass raw bytes from the original document return outi.UnmarshalTOML(value)
return outi.UnmarshalTOML(d.p.Raw(value.Raw))
} }
} }
} }
// Only try TextUnmarshaler for scalar types. For Array and InlineTable,
// fall through to struct/map unmarshaling to allow flexible unmarshaling
// where a type can implement UnmarshalText for string values but still
// be populated field-by-field from a table. See issue #974.
if value.Kind != unstable.Array && value.Kind != unstable.InlineTable {
ok, err := d.tryTextUnmarshaler(value, v) ok, err := d.tryTextUnmarshaler(value, v)
if ok || err != nil { if ok || err != nil {
return err return err
} }
}
switch value.Kind { switch value.Kind {
case unstable.String: case unstable.String:
@@ -921,9 +844,6 @@ func (d *decoder) unmarshalDateTime(value *unstable.Node, v reflect.Value) error
return err return err
} }
if v.Kind() != reflect.Interface && v.Type() != timeType {
return unstable.NewParserError(d.p.Raw(value.Raw), "%s", d.typeMismatchString("datetime", v.Type()))
}
v.Set(reflect.ValueOf(dt)) v.Set(reflect.ValueOf(dt))
return nil return nil
} }
@@ -934,14 +854,14 @@ func (d *decoder) unmarshalLocalDate(value *unstable.Node, v reflect.Value) erro
return err return err
} }
if v.Kind() != reflect.Interface && v.Type() != timeType {
return unstable.NewParserError(d.p.Raw(value.Raw), "%s", d.typeMismatchString("local date", v.Type()))
}
if v.Type() == timeType { if v.Type() == timeType {
v.Set(reflect.ValueOf(ld.AsTime(time.Local))) cast := ld.AsTime(time.Local)
v.Set(reflect.ValueOf(cast))
return nil return nil
} }
v.Set(reflect.ValueOf(ld)) v.Set(reflect.ValueOf(ld))
return nil return nil
} }
@@ -955,9 +875,6 @@ func (d *decoder) unmarshalLocalTime(value *unstable.Node, v reflect.Value) erro
return unstable.NewParserError(rest, "extra characters at the end of a local time") return unstable.NewParserError(rest, "extra characters at the end of a local time")
} }
if v.Kind() != reflect.Interface {
return unstable.NewParserError(d.p.Raw(value.Raw), "%s", d.typeMismatchString("local time", v.Type()))
}
v.Set(reflect.ValueOf(lt)) v.Set(reflect.ValueOf(lt))
return nil return nil
} }
@@ -972,14 +889,15 @@ func (d *decoder) unmarshalLocalDateTime(value *unstable.Node, v reflect.Value)
return unstable.NewParserError(rest, "extra characters at the end of a local date time") return unstable.NewParserError(rest, "extra characters at the end of a local date time")
} }
if v.Kind() != reflect.Interface && v.Type() != timeType {
return unstable.NewParserError(d.p.Raw(value.Raw), "%s", d.typeMismatchString("local datetime", v.Type()))
}
if v.Type() == timeType { if v.Type() == timeType {
v.Set(reflect.ValueOf(ldt.AsTime(time.Local))) cast := ldt.AsTime(time.Local)
v.Set(reflect.ValueOf(cast))
return nil return nil
} }
v.Set(reflect.ValueOf(ldt)) v.Set(reflect.ValueOf(ldt))
return nil return nil
} }
@@ -1034,9 +952,8 @@ const (
// compile time, so it is computed during initialization. // compile time, so it is computed during initialization.
var maxUint int64 = math.MaxInt64 var maxUint int64 = math.MaxInt64
func init() { //nolint:gochecknoinits func init() {
m := uint64(^uint(0)) m := uint64(^uint(0))
// #nosec G115
if m < uint64(maxUint) { if m < uint64(maxUint) {
maxUint = int64(m) maxUint = int64(m)
} }
@@ -1186,39 +1103,35 @@ func (d *decoder) keyFromData(keyType reflect.Type, data []byte) (reflect.Value,
return reflect.Value{}, fmt.Errorf("toml: error unmarshalling key type %s from text: %w", stringType, err) return reflect.Value{}, fmt.Errorf("toml: error unmarshalling key type %s from text: %w", stringType, err)
} }
return mk.Elem(), nil return mk.Elem(), nil
}
switch keyType.Kind() { case keyType.Kind() == reflect.Int || keyType.Kind() == reflect.Int8 || keyType.Kind() == reflect.Int16 || keyType.Kind() == reflect.Int32 || keyType.Kind() == reflect.Int64:
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
key, err := strconv.ParseInt(string(data), 10, 64) key, err := strconv.ParseInt(string(data), 10, 64)
if err != nil { if err != nil {
return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from integer: %w", stringType, err) return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from integer: %w", stringType, err)
} }
return reflect.ValueOf(key).Convert(keyType), nil return reflect.ValueOf(key).Convert(keyType), nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64: case keyType.Kind() == reflect.Uint || keyType.Kind() == reflect.Uint8 || keyType.Kind() == reflect.Uint16 || keyType.Kind() == reflect.Uint32 || keyType.Kind() == reflect.Uint64:
key, err := strconv.ParseUint(string(data), 10, 64) key, err := strconv.ParseUint(string(data), 10, 64)
if err != nil { if err != nil {
return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from unsigned integer: %w", stringType, err) return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from unsigned integer: %w", stringType, err)
} }
return reflect.ValueOf(key).Convert(keyType), nil return reflect.ValueOf(key).Convert(keyType), nil
case reflect.Float32: case keyType.Kind() == reflect.Float32:
key, err := strconv.ParseFloat(string(data), 32) key, err := strconv.ParseFloat(string(data), 32)
if err != nil { if err != nil {
return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from float: %w", stringType, err) return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from float: %w", stringType, err)
} }
return reflect.ValueOf(float32(key)), nil return reflect.ValueOf(float32(key)), nil
case reflect.Float64: case keyType.Kind() == reflect.Float64:
key, err := strconv.ParseFloat(string(data), 64) key, err := strconv.ParseFloat(string(data), 64)
if err != nil { if err != nil {
return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from float: %w", stringType, err) return reflect.Value{}, fmt.Errorf("toml: error parsing key of type %s from float: %w", stringType, err)
} }
return reflect.ValueOf(float64(key)), nil return reflect.ValueOf(float64(key)), nil
default:
return reflect.Value{}, fmt.Errorf("toml: cannot convert map key of type %s to expected type %s", stringType, keyType)
} }
return reflect.Value{}, fmt.Errorf("toml: cannot convert map key of type %s to expected type %s", stringType, keyType)
} }
func (d *decoder) handleKeyValuePart(key unstable.Iterator, value *unstable.Node, v reflect.Value) (reflect.Value, error) { func (d *decoder) handleKeyValuePart(key unstable.Iterator, value *unstable.Node, v reflect.Value) (reflect.Value, error) {
@@ -1270,8 +1183,7 @@ func (d *decoder) handleKeyValuePart(key unstable.Iterator, value *unstable.Node
if d.unmarshalerInterface { if d.unmarshalerInterface {
if v.CanAddr() && v.Addr().CanInterface() { if v.CanAddr() && v.Addr().CanInterface() {
if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok { if outi, ok := v.Addr().Interface().(unstable.Unmarshaler); ok {
// Pass raw bytes from the original document return reflect.Value{}, outi.UnmarshalTOML(value)
return reflect.Value{}, outi.UnmarshalTOML(d.p.Raw(value.Raw))
} }
} }
} }
@@ -1381,13 +1293,22 @@ func fieldByIndex(v reflect.Value, path []int) reflect.Value {
type fieldPathsMap = map[string][]int type fieldPathsMap = map[string][]int
var globalFieldPathsCache atomic.Value // map[reflect.Type]fieldPathsMap var globalFieldPathsCache atomic.Value // map[uintptr]fieldPathsMap
func structFieldPath(v reflect.Value, name string) ([]int, bool) { func structFieldPath(v reflect.Value, name string) ([]int, bool) {
t := v.Type() t := v.Type()
// reflect.Type is an interface. We want to use the address of the underlying
// rtype as the key.
// This avoids using the interface as map key, which is slower.
//
// In the future this should be replaced by t.Pointer() if it becomes available.
//
// v.Type() returns a reflect.Type interface.
// reflect.ValueOf(t).Pointer() returns the address of the rtype.
tid := reflect.ValueOf(t).Pointer()
cache, _ := globalFieldPathsCache.Load().(map[reflect.Type]fieldPathsMap) cache, _ := globalFieldPathsCache.Load().(map[uintptr]fieldPathsMap)
fieldPaths, ok := cache[t] fieldPaths, ok := cache[tid]
if !ok { if !ok {
fieldPaths = map[string][]int{} fieldPaths = map[string][]int{}
@@ -1398,8 +1319,8 @@ func structFieldPath(v reflect.Value, name string) ([]int, bool) {
fieldPaths[strings.ToLower(name)] = path fieldPaths[strings.ToLower(name)] = path
}) })
newCache := make(map[reflect.Type]fieldPathsMap, len(cache)+1) newCache := make(map[uintptr]fieldPathsMap, len(cache)+1)
newCache[t] = fieldPaths newCache[tid] = fieldPaths
for k, v := range cache { for k, v := range cache {
newCache[k] = v newCache[k] = v
} }
@@ -1423,9 +1344,7 @@ func forEachField(t reflect.Type, path []int, do func(name string, path []int))
continue continue
} }
fieldPath := make([]int, 0, len(path)+1) fieldPath := append(path, i)
fieldPath = append(fieldPath, path...)
fieldPath = append(fieldPath, i)
fieldPath = fieldPath[:len(fieldPath):len(fieldPath)] fieldPath = fieldPath[:len(fieldPath):len(fieldPath)]
name := f.Tag.Get("toml") name := f.Tag.Get("toml")
+53 -585
View File
@@ -11,9 +11,9 @@ import (
"testing" "testing"
"time" "time"
"git.ostiwe.com/ostiwe/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
"git.ostiwe.com/ostiwe/go-toml/v2/unstable" "github.com/pelletier/go-toml/v2/unstable"
) )
type unmarshalTextKey struct { type unmarshalTextKey struct {
@@ -33,8 +33,8 @@ func (k *unmarshalTextKey) UnmarshalText(text []byte) error {
type unmarshalBadTextKey struct{} type unmarshalBadTextKey struct{}
func (k *unmarshalBadTextKey) UnmarshalText([]byte) error { func (k *unmarshalBadTextKey) UnmarshalText(text []byte) error {
return errors.New("error") return fmt.Errorf("error")
} }
func ExampleDecoder_DisallowUnknownFields() { func ExampleDecoder_DisallowUnknownFields() {
@@ -96,136 +96,10 @@ func ExampleUnmarshal() {
// tags: [go toml] // tags: [go toml]
} }
// pluginConfig demonstrates how to implement dynamic unmarshaling
// based on a "type" field. This pattern is useful for plugin systems
// or polymorphic configuration.
type pluginConfig struct {
Type string
Config any
}
func (p *pluginConfig) UnmarshalTOML(data []byte) error {
// First, decode just the type field
var typeOnly struct {
Type string `toml:"type"`
}
if err := toml.Unmarshal(data, &typeOnly); err != nil {
return err
}
p.Type = typeOnly.Type
// Now decode the config based on the type
switch typeOnly.Type {
case "database":
var cfg struct {
Type string `toml:"type"`
Host string `toml:"host"`
Port int `toml:"port"`
}
if err := toml.Unmarshal(data, &cfg); err != nil {
return err
}
p.Config = map[string]any{"host": cfg.Host, "port": cfg.Port}
case "cache":
var cfg struct {
Type string `toml:"type"`
TTL int `toml:"ttl"`
}
if err := toml.Unmarshal(data, &cfg); err != nil {
return err
}
p.Config = map[string]any{"ttl": cfg.TTL}
}
return nil
}
// This example demonstrates dynamic unmarshaling based on a discriminator
// field. The pluginConfig type uses UnmarshalTOML to first read the "type"
// field, then decode the rest of the configuration based on that type.
// This pattern is useful for plugin systems or configuration that varies
// by type.
func ExampleDecoder_EnableUnmarshalerInterface_dynamicConfig() {
doc := `
[[plugins]]
type = "database"
host = "localhost"
port = 5432
[[plugins]]
type = "cache"
ttl = 300
`
type Config struct {
Plugins []pluginConfig `toml:"plugins"`
}
var cfg Config
err := toml.NewDecoder(strings.NewReader(doc)).
EnableUnmarshalerInterface().
Decode(&cfg)
if err != nil {
panic(err)
}
for _, p := range cfg.Plugins {
fmt.Printf("type=%s config=%v\n", p.Type, p.Config)
}
// Output:
// type=database config=map[host:localhost port:5432]
// type=cache config=map[ttl:300]
}
// This example demonstrates using RawMessage to capture raw TOML bytes
// for later processing. RawMessage is similar to json.RawMessage - it
// delays decoding so you can inspect the raw content or decode it
// differently based on context.
func ExampleDecoder_EnableUnmarshalerInterface_rawMessage() {
doc := `
[plugin]
name = "example"
version = "1.0"
enabled = true
`
type Config struct {
Plugin unstable.RawMessage `toml:"plugin"`
}
var cfg Config
err := toml.NewDecoder(strings.NewReader(doc)).
EnableUnmarshalerInterface().
Decode(&cfg)
if err != nil {
panic(err)
}
// cfg.Plugin contains the raw TOML bytes
fmt.Printf("Raw TOML captured:\n%s", cfg.Plugin)
// You can later decode it into a specific type
var plugin struct {
Name string `toml:"name"`
Version string `toml:"version"`
Enabled bool `toml:"enabled"`
}
if err := toml.Unmarshal(cfg.Plugin, &plugin); err != nil {
panic(err)
}
fmt.Printf("Decoded: name=%s version=%s enabled=%v\n",
plugin.Name, plugin.Version, plugin.Enabled)
// Output:
// Raw TOML captured:
// name = "example"
// version = "1.0"
// enabled = true
// Decoded: name=example version=1.0 enabled=true
}
type badReader struct{} type badReader struct{}
func (r *badReader) Read([]byte) (int, error) { func (r *badReader) Read([]byte) (int, error) {
return 0, errors.New("testing error") return 0, fmt.Errorf("testing error")
} }
func TestDecodeReaderError(t *testing.T) { func TestDecodeReaderError(t *testing.T) {
@@ -237,6 +111,7 @@ func TestDecodeReaderError(t *testing.T) {
assert.Error(t, err) assert.Error(t, err)
} }
// nolint:funlen
func TestUnmarshal_Integers(t *testing.T) { func TestUnmarshal_Integers(t *testing.T) {
examples := []struct { examples := []struct {
desc string desc string
@@ -320,6 +195,7 @@ func TestUnmarshal_Integers(t *testing.T) {
} }
} }
//nolint:funlen
func TestUnmarshal_Floats(t *testing.T) { func TestUnmarshal_Floats(t *testing.T) {
examples := []struct { examples := []struct {
desc string desc string
@@ -457,6 +333,7 @@ func TestUnmarshal_Floats(t *testing.T) {
} }
} }
//nolint:funlen
func TestUnmarshal(t *testing.T) { func TestUnmarshal(t *testing.T) {
type test struct { type test struct {
target interface{} target interface{}
@@ -533,7 +410,6 @@ foo = "bar"`,
target: &doc{}, target: &doc{},
expected: &doc{{A: "a", B: "1"}: "foo"}, expected: &doc{{A: "a", B: "1"}: "foo"},
assert: func(t *testing.T, test test) { assert: func(t *testing.T, test test) {
t.Helper()
// Despite the documentation: // Despite the documentation:
// Pointer variable equality is determined based on the equality of the // Pointer variable equality is determined based on the equality of the
// referenced values (as opposed to the memory addresses). // referenced values (as opposed to the memory addresses).
@@ -1470,7 +1346,7 @@ B = "data"`,
input: `foo = "bar"`, input: `foo = "bar"`,
gen: func() test { gen: func() test {
type doc struct { type doc struct {
foo string //nolint:unused foo string
} }
return test{ return test{
target: &doc{}, target: &doc{},
@@ -2063,6 +1939,9 @@ B = "data"`,
return test{ return test{
target: &map[int]string{}, target: &map[int]string{},
expected: &map[int]string{1: "a"}, expected: &map[int]string{1: "a"},
assert: func(t *testing.T, test test) {
assert.Equal(t, test.expected, test.target)
},
} }
}, },
}, },
@@ -2073,6 +1952,9 @@ B = "data"`,
return test{ return test{
target: &map[int8]string{}, target: &map[int8]string{},
expected: &map[int8]string{1: "a"}, expected: &map[int8]string{1: "a"},
assert: func(t *testing.T, test test) {
assert.Equal(t, test.expected, test.target)
},
} }
}, },
}, },
@@ -2083,6 +1965,9 @@ B = "data"`,
return test{ return test{
target: &map[int64]string{}, target: &map[int64]string{},
expected: &map[int64]string{1: "a"}, expected: &map[int64]string{1: "a"},
assert: func(t *testing.T, test test) {
assert.Equal(t, test.expected, test.target)
},
} }
}, },
}, },
@@ -2093,6 +1978,9 @@ B = "data"`,
return test{ return test{
target: &map[uint]string{}, target: &map[uint]string{},
expected: &map[uint]string{1: "a"}, expected: &map[uint]string{1: "a"},
assert: func(t *testing.T, test test) {
assert.Equal(t, test.expected, test.target)
},
} }
}, },
}, },
@@ -2103,6 +1991,9 @@ B = "data"`,
return test{ return test{
target: &map[uint8]string{}, target: &map[uint8]string{},
expected: &map[uint8]string{1: "a"}, expected: &map[uint8]string{1: "a"},
assert: func(t *testing.T, test test) {
assert.Equal(t, test.expected, test.target)
},
} }
}, },
}, },
@@ -2113,6 +2004,9 @@ B = "data"`,
return test{ return test{
target: &map[uint64]string{}, target: &map[uint64]string{},
expected: &map[uint64]string{1: "a"}, expected: &map[uint64]string{1: "a"},
assert: func(t *testing.T, test test) {
assert.Equal(t, test.expected, test.target)
},
} }
}, },
}, },
@@ -2133,6 +2027,9 @@ B = "data"`,
return test{ return test{
target: &map[float64]string{}, target: &map[float64]string{},
expected: &map[float64]string{1.01: "a"}, expected: &map[float64]string{1.01: "a"},
assert: func(t *testing.T, test test) {
assert.Equal(t, test.expected, test.target)
},
} }
}, },
}, },
@@ -2153,6 +2050,9 @@ B = "data"`,
return test{ return test{
target: &map[float32]string{}, target: &map[float32]string{},
expected: &map[float32]string{1.01: "a"}, expected: &map[float32]string{1.01: "a"},
assert: func(t *testing.T, test test) {
assert.Equal(t, test.expected, test.target)
},
} }
}, },
}, },
@@ -2304,8 +2204,7 @@ port = "bad"
err := toml.NewDecoder(file).Decode(&cfg) err := toml.NewDecoder(file).Decode(&cfg)
assert.Error(t, err) assert.Error(t, err)
var x *toml.DecodeError x := err.(*toml.DecodeError)
assert.True(t, errors.As(err, &x))
assert.Equal(t, "toml: cannot decode TOML string into struct field toml_test.Server.Port of type int", x.Error()) assert.Equal(t, "toml: cannot decode TOML string into struct field toml_test.Server.Port of type int", x.Error())
expected := `1| [server] expected := `1| [server]
2| path = "/my/path" 2| path = "/my/path"
@@ -2336,8 +2235,7 @@ port = 50
err := toml.NewDecoder(file).Decode(&cfg) err := toml.NewDecoder(file).Decode(&cfg)
assert.Error(t, err) assert.Error(t, err)
var x *toml.DecodeError x := err.(*toml.DecodeError)
assert.True(t, errors.As(err, &x))
assert.Equal(t, "toml: cannot decode TOML integer into struct field toml_test.Server.Path of type string", x.Error()) assert.Equal(t, "toml: cannot decode TOML integer into struct field toml_test.Server.Path of type string", x.Error())
expected := `1| [server] expected := `1| [server]
2| path = 100 2| path = 100
@@ -2590,7 +2488,7 @@ func TestIssue508(t *testing.T) {
t1 := text{} t1 := text{}
err := toml.Unmarshal(b, &t1) err := toml.Unmarshal(b, &t1)
assert.NoError(t, err) assert.NoError(t, err)
assert.Equal(t, "This is a title", t1.Title) assert.Equal(t, "This is a title", t1.head.Title)
} }
func TestIssue507(t *testing.T) { func TestIssue507(t *testing.T) {
@@ -2602,7 +2500,7 @@ func TestIssue507(t *testing.T) {
type uuid [16]byte type uuid [16]byte
func (u *uuid) UnmarshalText([]byte) (err error) { func (u *uuid) UnmarshalText(text []byte) (err error) {
// Note: the original reported issue had a more complex implementation // Note: the original reported issue had a more complex implementation
// of this function. But the important part is to verify that a // of this function. But the important part is to verify that a
// non-struct type implementing UnmarshalText works with the unmarshal // non-struct type implementing UnmarshalText works with the unmarshal
@@ -2645,7 +2543,7 @@ xz_hash = "1a48f723fea1f17d786ce6eadd9d00914d38062d28fd9c455ed3c3801905b388"
`) `)
type target struct { type target struct {
XZ_URL string //revive:disable:var-naming XZ_URL string
} }
type pkg struct { type pkg struct {
@@ -2896,7 +2794,7 @@ func TestIssue772(t *testing.T) {
config := Config{} config := Config{}
err := toml.Unmarshal(defaultConfigFile, &config) err := toml.Unmarshal(defaultConfigFile, &config)
assert.NoError(t, err) assert.NoError(t, err)
assert.Equal(t, "reach-masterdev-", config.FilePattern) assert.Equal(t, "reach-masterdev-", config.FileHandling.FilePattern)
} }
func TestIssue774(t *testing.T) { func TestIssue774(t *testing.T) {
@@ -3056,7 +2954,7 @@ blah.a = "def"`)
assert.NoError(t, err) assert.NoError(t, err)
assert.Equal(t, "abc", cfg.Fizz) assert.Equal(t, "abc", cfg.Fizz)
assert.Equal(t, "def", cfg.A) assert.Equal(t, "def", cfg.blah.A)
assert.Equal(t, "def", cfg.A) assert.Equal(t, "def", cfg.A)
} }
@@ -3286,17 +3184,7 @@ world'`,
{ {
desc: "invalid seconds value", desc: "invalid seconds value",
data: `a=1979-05-27T12:45:99`, data: `a=1979-05-27T12:45:99`,
msg: `seconds cannot be greater than 59`, msg: `seconds cannot be greater 60`,
},
{
desc: "leap second not supported",
data: `a=1979-05-27T12:45:60`,
msg: `seconds cannot be greater than 59`,
},
{
desc: "leap second with max date causes overflow",
data: `s=9999-12-31 23:59:60z`,
msg: `seconds cannot be greater than 59`,
}, },
{ {
desc: `binary with invalid digit`, desc: `binary with invalid digit`,
@@ -3596,7 +3484,7 @@ world'`,
func TestOmitEmpty(t *testing.T) { func TestOmitEmpty(t *testing.T) {
type inner struct { type inner struct {
private string //nolint:unused private string
Skip string `toml:"-"` Skip string `toml:"-"`
V string V string
} }
@@ -3712,6 +3600,7 @@ func TestASCIIControlCharacters(t *testing.T) {
} }
} }
//nolint:funlen
func TestLocalDateTime(t *testing.T) { func TestLocalDateTime(t *testing.T) {
examples := []struct { examples := []struct {
desc string desc string
@@ -4026,10 +3915,10 @@ type CustomUnmarshalerKey struct {
A int64 A int64
} }
func (k *CustomUnmarshalerKey) UnmarshalTOML(data []byte) error { func (k *CustomUnmarshalerKey) UnmarshalTOML(value *unstable.Node) error {
item, err := strconv.ParseInt(string(data), 10, 64) item, err := strconv.ParseInt(string(value.Data), 10, 64)
if err != nil { if err != nil {
return fmt.Errorf("error converting to int64, %w", err) return fmt.Errorf("error converting to int64, %v", err)
} }
k.A = item k.A = item
return nil return nil
@@ -4115,7 +4004,7 @@ foo = "bar"`,
type doc994 struct{} type doc994 struct{}
func (d *doc994) UnmarshalTOML([]byte) error { func (d *doc994) UnmarshalTOML(value *unstable.Node) error {
return errors.New("expected-error") return errors.New("expected-error")
} }
@@ -4138,8 +4027,8 @@ type doc994ok struct {
S string S string
} }
func (d *doc994ok) UnmarshalTOML(data []byte) error { func (d *doc994ok) UnmarshalTOML(value *unstable.Node) error {
d.S = string(data) + " from unmarshaler" d.S = string(value.Data) + " from unmarshaler"
return nil return nil
} }
@@ -4152,8 +4041,7 @@ func TestIssue994_OK(t *testing.T) {
Decode(&d) Decode(&d)
assert.NoError(t, err) assert.NoError(t, err)
// With bytes-based interface, raw TOML bytes are passed including quotes assert.Equal(t, "bar from unmarshaler", d.S)
assert.Equal(t, "\"bar\" from unmarshaler", d.S)
} }
func TestIssue995(t *testing.T) { func TestIssue995(t *testing.T) {
@@ -4349,428 +4237,8 @@ func TestIssue995_SliceNonEmpty_UsesLastElement(t *testing.T) {
} }
assert.Equal(t, 2, len(r.Rules[0].Allowlists)) assert.Equal(t, 2, len(r.Rules[0].Allowlists))
// Values presence check // Values presence check
got := [...]string{r.Rules[0].Allowlists[0].Description, r.Rules[0].Allowlists[1].Description} got := []string{r.Rules[0].Allowlists[0].Description, r.Rules[0].Allowlists[1].Description}
if got != [2]string{"a", "b"} && got != [2]string{"b", "a"} { if !(got[0] == "a" && got[1] == "b") && !(got[0] == "b" && got[1] == "a") {
t.Fatalf("unexpected values in allowlists: %v", got) t.Fatalf("unexpected values in allowlists: %v", got)
} }
} }
// fooConfig974 is a struct that implements UnmarshalText for simple string
// parsing, but can also be populated field-by-field from a TOML table.
type fooConfig974 struct {
Name string `toml:"name"`
Value string `toml:"value"`
}
func (f *fooConfig974) UnmarshalText(text []byte) error {
s := string(text)
f.Name = s
f.Value = s
return nil
}
type config974 struct {
Foo []fooConfig974 `toml:"foo"`
}
func TestIssue974_UnmarshalTextFallbackToStructForInlineTable(t *testing.T) {
// When the TOML value is an inline table, the unmarshaler should skip
// UnmarshalText and populate the struct fields directly.
doc := `foo = [{name = "a", value = "a"}, {name = "b", value = "b"}]`
var cfg config974
err := toml.Unmarshal([]byte(doc), &cfg)
assert.NoError(t, err)
assert.Equal(t, config974{
Foo: []fooConfig974{
{Name: "a", Value: "a"},
{Name: "b", Value: "b"},
},
}, cfg)
}
func TestIssue974_UnmarshalTextStillWorksForStrings(t *testing.T) {
// When the TOML value is a string, UnmarshalText should still be used.
doc := `foo = ["a", "b"]`
var cfg config974
err := toml.Unmarshal([]byte(doc), &cfg)
assert.NoError(t, err)
assert.Equal(t, config974{
Foo: []fooConfig974{
{Name: "a", Value: "a"},
{Name: "b", Value: "b"},
},
}, cfg)
}
// singleFooConfig974 tests the inline table case for a single value (not array)
type singleConfig974 struct {
Foo fooConfig974 `toml:"foo"`
}
func TestIssue974_SingleInlineTable(t *testing.T) {
// A single inline table should also skip UnmarshalText
doc := `foo = {name = "hello", value = "world"}`
var cfg singleConfig974
err := toml.Unmarshal([]byte(doc), &cfg)
assert.NoError(t, err)
assert.Equal(t, singleConfig974{
Foo: fooConfig974{Name: "hello", Value: "world"},
}, cfg)
}
func TestIssue974_SingleString(t *testing.T) {
// A single string should use UnmarshalText
doc := `foo = "hello"`
var cfg singleConfig974
err := toml.Unmarshal([]byte(doc), &cfg)
assert.NoError(t, err)
assert.Equal(t, singleConfig974{
Foo: fooConfig974{Name: "hello", Value: "hello"},
}, cfg)
}
func TestIssue974_TableSyntax(t *testing.T) {
// Regular table syntax should also work (uses struct unmarshaling)
doc := `
[foo]
name = "hello"
value = "world"
`
var cfg singleConfig974
err := toml.Unmarshal([]byte(doc), &cfg)
assert.NoError(t, err)
assert.Equal(t, singleConfig974{
Foo: fooConfig974{Name: "hello", Value: "world"},
}, cfg)
}
func TestIssue974_ArrayTableSyntax(t *testing.T) {
// Array of tables syntax should also work
doc := `
[[foo]]
name = "a"
value = "a"
[[foo]]
name = "b"
value = "b"
`
var cfg config974
err := toml.Unmarshal([]byte(doc), &cfg)
assert.NoError(t, err)
assert.Equal(t, config974{
Foo: []fooConfig974{
{Name: "a", Value: "a"},
{Name: "b", Value: "b"},
},
}, cfg)
}
func TestIssue1028(t *testing.T) {
// Datetime values assigned to incompatible types should return an error,
// not panic.
type Item struct {
Name string `toml:"name"`
}
type Config struct {
Items map[string]Item `toml:"items"`
}
// Error: "cannot decode TOML datetime into struct field Config.Items of type map[string]Item"
t.Run("OffsetDateTime", func(t *testing.T) {
var c Config
err := toml.Unmarshal([]byte(`items = 2023-01-01T10:20:30Z`), &c)
assert.Error(t, err)
})
// Error: "cannot decode TOML local datetime into struct field Config.Items of type map[string]Item"
t.Run("LocalDateTime", func(t *testing.T) {
var c Config
err := toml.Unmarshal([]byte(`items = 2023-01-01T10:20:30`), &c)
assert.Error(t, err)
})
// Error: "cannot decode TOML local date into struct field Config.Items of type map[string]Item"
t.Run("LocalDate", func(t *testing.T) {
var c Config
err := toml.Unmarshal([]byte(`items = 2023-01-01`), &c)
assert.Error(t, err)
})
// Error: "cannot decode TOML local time into struct field Config.Items of type map[string]Item"
t.Run("LocalTime", func(t *testing.T) {
var c Config
err := toml.Unmarshal([]byte(`items = 10:20:30`), &c)
assert.Error(t, err)
})
}
// Tests for issue #873 - Bring back toml.Unmarshaler for tables and arrays
type customTable873 struct {
Keys []string
Values map[string]string
}
func (c *customTable873) UnmarshalTOML(data []byte) error {
c.Keys = []string{}
c.Values = make(map[string]string)
// Parse the raw TOML bytes into a map to extract keys in order
// For this test, we use a simple line-by-line parser to preserve order
lines := bytes.Split(data, []byte{'\n'})
for _, line := range lines {
line = bytes.TrimSpace(line)
if len(line) == 0 {
continue
}
// Skip table headers
if line[0] == '[' {
continue
}
// Parse key = value
eqIdx := bytes.Index(line, []byte{'='})
if eqIdx < 0 {
continue
}
key := string(bytes.TrimSpace(line[:eqIdx]))
// Remove quotes from quoted keys
if len(key) >= 2 && key[0] == '"' && key[len(key)-1] == '"' {
key = key[1 : len(key)-1]
}
valueBytes := bytes.TrimSpace(line[eqIdx+1:])
// Remove quotes from string values
if len(valueBytes) >= 2 && valueBytes[0] == '"' && valueBytes[len(valueBytes)-1] == '"' {
valueBytes = valueBytes[1 : len(valueBytes)-1]
}
c.Keys = append(c.Keys, key)
c.Values[key] = string(valueBytes)
}
return nil
}
// Test for split tables - when the same parent table is defined in multiple places
// This is a key requirement for issue #873: if type A implements Unmarshaler,
// and [a.b] and [a.d] are defined with another table [x] in between,
// A should receive content for both b and d, but not x.
func TestIssue873_SplitTables(t *testing.T) {
// For this test, we expect each sub-table to be handled separately
// The parent doesn't receive the sub-tables directly - each sub-table
// (b and d) gets its own call to handleKeyValues
type Config struct {
A struct {
B customTable873 `toml:"b"`
D customTable873 `toml:"d"`
} `toml:"a"`
X customTable873 `toml:"x"`
}
doc := `
[a.b]
C = "1"
[x]
Y = "100"
[a.d]
E = "2"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
// Each sub-table should have received its own key-values
assert.Equal(t, []string{"C"}, cfg.A.B.Keys)
assert.Equal(t, "1", cfg.A.B.Values["C"])
assert.Equal(t, []string{"E"}, cfg.A.D.Keys)
assert.Equal(t, "2", cfg.A.D.Values["E"])
assert.Equal(t, []string{"Y"}, cfg.X.Keys)
assert.Equal(t, "100", cfg.X.Values["Y"])
}
// Test using RawMessage to capture raw TOML bytes
func TestIssue873_RawMessage(t *testing.T) {
type Config struct {
Plugin unstable.RawMessage `toml:"plugin"`
}
doc := `
[plugin]
name = "example"
version = "1.0"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
// RawMessage should contain the raw key-value bytes
expected := "name = \"example\"\nversion = \"1.0\"\n"
assert.Equal(t, expected, string(cfg.Plugin))
}
// Test keys that need quoting (contain special characters)
func TestIssue873_QuotedKeys(t *testing.T) {
type Config struct {
Section customTable873 `toml:"section"`
}
doc := `
[section]
"key with spaces" = "value1"
"key.with.dots" = "value2"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, 2, len(cfg.Section.Keys))
assert.Equal(t, "value1", cfg.Section.Values["key with spaces"])
assert.Equal(t, "value2", cfg.Section.Values["key.with.dots"])
}
// errorUnmarshaler873 is used to test error propagation from UnmarshalTOML
type errorUnmarshaler873 struct{}
func (e *errorUnmarshaler873) UnmarshalTOML([]byte) error {
return errors.New("intentional error")
}
// Test error propagation from UnmarshalTOML
func TestIssue873_UnmarshalerError(t *testing.T) {
doc := `
[section]
key = "value"
`
type Config struct {
Section errorUnmarshaler873 `toml:"section"`
}
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.Error(t, err)
assert.True(t, strings.Contains(err.Error(), "intentional error"))
}
// Test dotted keys in a table (e.g., a.b = value)
func TestIssue873_DottedKeys(t *testing.T) {
type Config struct {
Section customTable873 `toml:"section"`
}
doc := `
[section]
sub.key = "value1"
another.nested.key = "value2"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.Equal(t, 2, len(cfg.Section.Keys))
// The dotted keys should be preserved in the raw output
assert.Equal(t, "value1", cfg.Section.Values["sub.key"])
assert.Equal(t, "value2", cfg.Section.Values["another.nested.key"])
}
// Test pointer to pointer to Unmarshaler (covers pointer dereferencing loop)
func TestIssue873_DoublePointerUnmarshaler(t *testing.T) {
type Config struct {
Section **customTable873 `toml:"section"`
}
doc := `
[section]
key = "value"
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.True(t, cfg.Section != nil)
assert.True(t, *cfg.Section != nil)
assert.Equal(t, []string{"key"}, (*cfg.Section).Keys)
assert.Equal(t, "value", (*cfg.Section).Values["key"])
}
// formattingCapture captures the raw TOML bytes to verify formatting preservation
type formattingCapture struct {
RawBytes string
}
func (f *formattingCapture) UnmarshalTOML(data []byte) error {
f.RawBytes = string(data)
return nil
}
func TestIssue873_FormattingPreservation(t *testing.T) {
type Config struct {
Section *formattingCapture `toml:"section"`
}
// Test that various formatting styles are preserved:
// - Extra spaces around '='
// - Literal strings (single quotes)
// - Hex numbers
// - Inline tables
doc := `[section]
key1 = "value with spaces"
key2 = 'literal string'
hex_val = 0xDEADBEEF
inline = { a = 1, b = 2 }
`
var cfg Config
err := toml.NewDecoder(bytes.NewReader([]byte(doc))).
EnableUnmarshalerInterface().
Decode(&cfg)
assert.NoError(t, err)
assert.True(t, cfg.Section != nil)
// The raw bytes should preserve original formatting
raw := cfg.Section.RawBytes
// Check that extra spaces around '=' are preserved
assert.True(t, strings.Contains(raw, "key1 = \"value with spaces\""),
"Expected spacing to be preserved, got: %s", raw)
// Check that literal string style is preserved
assert.True(t, strings.Contains(raw, "key2 = 'literal string'"),
"Expected literal string to be preserved, got: %s", raw)
// Check that hex format is preserved
assert.True(t, strings.Contains(raw, "hex_val = 0xDEADBEEF"),
"Expected hex format to be preserved, got: %s", raw)
// Check that inline table is preserved
assert.True(t, strings.Contains(raw, "inline = { a = 1, b = 2 }"),
"Expected inline table to be preserved, got: %s", raw)
}
+17 -45
View File
@@ -1,7 +1,6 @@
package unstable package unstable
import ( import (
"errors"
"fmt" "fmt"
) )
@@ -17,43 +16,30 @@ import (
// // do something with n // // do something with n
// } // }
type Iterator struct { type Iterator struct {
nodes *[]Node
idx int32
started bool started bool
node *Node
} }
// Next moves the iterator forward and returns true if points to a // Next moves the iterator forward and returns true if points to a
// node, false otherwise. // node, false otherwise.
func (c *Iterator) Next() bool { func (c *Iterator) Next() bool {
if c.nodes == nil {
return false
}
nodes := *c.nodes
if !c.started { if !c.started {
c.started = true c.started = true
} else { } else if c.node.Valid() {
idx := c.idx c.node = c.node.Next()
if idx >= 0 && int(idx) < len(nodes) {
c.idx = nodes[idx].next
} }
} return c.node.Valid()
return c.idx >= 0 && int(c.idx) < len(nodes)
} }
// IsLast returns true if the current node of the iterator is the last // IsLast returns true if the current node of the iterator is the last
// one. Subsequent calls to Next() will return false. // one. Subsequent calls to Next() will return false.
func (c *Iterator) IsLast() bool { func (c *Iterator) IsLast() bool {
return c.nodes == nil || c.idx < 0 || (*c.nodes)[c.idx].next < 0 return c.node.next == nil
} }
// Node returns a pointer to the node pointed at by the iterator. // Node returns a pointer to the node pointed at by the iterator.
func (c *Iterator) Node() *Node { func (c *Iterator) Node() *Node {
if c.nodes == nil || c.idx < 0 { return c.node
return nil
}
n := &(*c.nodes)[c.idx]
n.nodes = c.nodes
return n
} }
// Node in a TOML expression AST. // Node in a TOML expression AST.
@@ -76,12 +62,9 @@ type Node struct {
Raw Range // Raw bytes from the input. Raw Range // Raw bytes from the input.
Data []byte // Node value (either allocated or referencing the input). Data []byte // Node value (either allocated or referencing the input).
// Absolute indices into the backing nodes slice. -1 means none. // References to other nodes.
next int32 next *Node // nil if last element
child int32 child *Node // nil if no child
// Reference to the backing nodes slice for navigation.
nodes *[]Node
} }
// Range of bytes in the document. // Range of bytes in the document.
@@ -92,24 +75,14 @@ type Range struct {
// Next returns a pointer to the next node, or nil if there is no next node. // Next returns a pointer to the next node, or nil if there is no next node.
func (n *Node) Next() *Node { func (n *Node) Next() *Node {
if n.next < 0 { return n.next
return nil
}
next := &(*n.nodes)[n.next]
next.nodes = n.nodes
return next
} }
// Child returns a pointer to the first child node of this node. Other children // Child returns a pointer to the first child node of this node. Other children
// can be accessed calling Next on the first child. Returns nil if this Node // can be accessed calling Next on the first child. Returns nil if this Node
// has no child. // has no child.
func (n *Node) Child() *Node { func (n *Node) Child() *Node {
if n.child < 0 { return n.child
return nil
}
child := &(*n.nodes)[n.child]
child.nodes = n.nodes
return child
} }
// Valid returns true if the node's kind is set (not to Invalid). // Valid returns true if the node's kind is set (not to Invalid).
@@ -123,14 +96,13 @@ func (n *Node) Valid() bool {
func (n *Node) Key() Iterator { func (n *Node) Key() Iterator {
switch n.Kind { switch n.Kind {
case KeyValue: case KeyValue:
child := n.child value := n.Child()
if child < 0 { if !value.Valid() {
panic(errors.New("KeyValue should have at least two children")) panic(fmt.Errorf("KeyValue should have at least two children"))
} }
valueNode := &(*n.nodes)[child] return Iterator{node: value.Next()}
return Iterator{nodes: n.nodes, idx: valueNode.next}
case Table, ArrayTable: case Table, ArrayTable:
return Iterator{nodes: n.nodes, idx: n.child} return Iterator{node: n.Child()}
default: default:
panic(fmt.Errorf("Key() is not supported on a %s", n.Kind)) panic(fmt.Errorf("Key() is not supported on a %s", n.Kind))
} }
@@ -145,5 +117,5 @@ func (n *Node) Value() *Node {
// Children returns an iterator over a node's children. // Children returns an iterator over a node's children.
func (n *Node) Children() Iterator { func (n *Node) Children() Iterator {
return Iterator{nodes: n.nodes, idx: n.child} return Iterator{node: n.Child()}
} }
+14 -16
View File
@@ -5,14 +5,12 @@ import (
"testing" "testing"
) )
var ( var valid10Ascii = []byte("1234567890")
valid10ASCII = []byte("1234567890") var valid10Utf8 = []byte("日本語a")
valid10Utf8 = []byte("日本語a") var valid1kUtf8 = bytes.Repeat([]byte("0123456789日本語日本語日本語日abcdefghijklmnopqrstuvwx"), 16)
valid1kUtf8 = bytes.Repeat([]byte("0123456789日本語日本語日本語日abcdefghijklmnopqrstuvwx"), 16) var valid1MUtf8 = bytes.Repeat(valid1kUtf8, 1024)
valid1MUtf8 = bytes.Repeat(valid1kUtf8, 1024) var valid1kAscii = bytes.Repeat([]byte("012345678998jhjklasDJKLAAdjdfjsdklfjdslkabcdefghijklmnopqrstuvwx"), 16)
valid1kASCII = bytes.Repeat([]byte("012345678998jhjklasDJKLAAdjdfjsdklfjdslkabcdefghijklmnopqrstuvwx"), 16) var valid1MAscii = bytes.Repeat(valid1kAscii, 1024)
valid1MASCII = bytes.Repeat(valid1kASCII, 1024)
)
func BenchmarkScanComments(b *testing.B) { func BenchmarkScanComments(b *testing.B) {
wrap := func(x []byte) []byte { wrap := func(x []byte) []byte {
@@ -20,9 +18,9 @@ func BenchmarkScanComments(b *testing.B) {
} }
inputs := map[string][]byte{ inputs := map[string][]byte{
"10Valid": wrap(valid10ASCII), "10Valid": wrap(valid10Ascii),
"1kValid": wrap(valid1kASCII), "1kValid": wrap(valid1kAscii),
"1MValid": wrap(valid1MASCII), "1MValid": wrap(valid1MAscii),
"10ValidUtf8": wrap(valid10Utf8), "10ValidUtf8": wrap(valid10Utf8),
"1kValidUtf8": wrap(valid1kUtf8), "1kValidUtf8": wrap(valid1kUtf8),
"1MValidUtf8": wrap(valid1MUtf8), "1MValidUtf8": wrap(valid1MUtf8),
@@ -35,7 +33,7 @@ func BenchmarkScanComments(b *testing.B) {
b.ResetTimer() b.ResetTimer()
for i := 0; i < b.N; i++ { for i := 0; i < b.N; i++ {
_, _, _ = scanComment(input) scanComment(input)
} }
}) })
} }
@@ -47,9 +45,9 @@ func BenchmarkParseLiteralStringValid(b *testing.B) {
} }
inputs := map[string][]byte{ inputs := map[string][]byte{
"10Valid": wrap(valid10ASCII), "10Valid": wrap(valid10Ascii),
"1kValid": wrap(valid1kASCII), "1kValid": wrap(valid1kAscii),
"1MValid": wrap(valid1MASCII), "1MValid": wrap(valid1MAscii),
"10ValidUtf8": wrap(valid10Utf8), "10ValidUtf8": wrap(valid10Utf8),
"1kValidUtf8": wrap(valid1kUtf8), "1kValidUtf8": wrap(valid1kUtf8),
"1MValidUtf8": wrap(valid1MUtf8), "1MValidUtf8": wrap(valid1MUtf8),
@@ -65,7 +63,7 @@ func BenchmarkParseLiteralStringValid(b *testing.B) {
for i := 0; i < b.N; i++ { for i := 0; i < b.N; i++ {
_, _, _, err := p.parseLiteralString(input) _, _, _, err := p.parseLiteralString(input)
if err != nil { if err != nil {
b.Error(err) panic(err)
} }
} }
}) })
+85 -28
View File
@@ -4,61 +4,118 @@ package unstable
// //
// It is immutable once constructed with Builder. // It is immutable once constructed with Builder.
type root struct { type root struct {
nodes []Node first *Node
} }
func (r *root) at(idx reference) *Node { // Iterator over the top level nodes.
return &r.nodes[idx] func (r *root) Iterator() Iterator {
return Iterator{node: r.first}
} }
type reference int type reference struct {
*Node
}
const invalidReference reference = -1 var invalidReference = reference{}
func (r reference) Valid() bool { func (r reference) Valid() bool {
return r != invalidReference return r.Node != nil
} }
type builder struct { type builder struct {
tree root // chunks of nodes. Pointers to nodes are stable because we only append
lastIdx int // to the last chunk, and chunks are allocated with fixed capacity.
chunks [][]Node
// current chunk index
chunkIdx int
// root node of the tree
root root
// last pushed node (for chaining)
last *Node
}
const initialChunkSize = 16
const maxChunkSize = 2048
func (b *builder) Tree() *root {
return &b.root
} }
func (b *builder) NodeAt(ref reference) *Node { func (b *builder) NodeAt(ref reference) *Node {
n := b.tree.at(ref) return ref.Node
n.nodes = &b.tree.nodes
return n
} }
func (b *builder) Reset() { func (b *builder) Reset() {
b.tree.nodes = b.tree.nodes[:0] b.chunkIdx = 0
b.lastIdx = 0 for i := range b.chunks {
b.chunks[i] = b.chunks[i][:0]
}
b.root.first = nil
b.last = nil
}
func (b *builder) ensureCapacity() {
if b.chunkIdx >= len(b.chunks) {
size := initialChunkSize
if len(b.chunks) > 0 {
lastCap := cap(b.chunks[len(b.chunks)-1])
size = lastCap * 2
if size > maxChunkSize {
size = maxChunkSize
}
}
b.chunks = append(b.chunks, make([]Node, 0, size))
}
if len(b.chunks[b.chunkIdx]) == cap(b.chunks[b.chunkIdx]) {
b.chunkIdx++
if b.chunkIdx >= len(b.chunks) {
size := initialChunkSize
if len(b.chunks) > 0 {
lastCap := cap(b.chunks[len(b.chunks)-1])
size = lastCap * 2
if size > maxChunkSize {
size = maxChunkSize
}
}
b.chunks = append(b.chunks, make([]Node, 0, size))
}
}
}
func (b *builder) push(n Node) *Node {
b.ensureCapacity()
chunk := &b.chunks[b.chunkIdx]
*chunk = append(*chunk, n)
return &(*chunk)[len(*chunk)-1]
} }
func (b *builder) Push(n Node) reference { func (b *builder) Push(n Node) reference {
b.lastIdx = len(b.tree.nodes) ptr := b.push(n)
n.next = -1 if b.root.first == nil {
n.child = -1 b.root.first = ptr
b.tree.nodes = append(b.tree.nodes, n) }
return reference(b.lastIdx) b.last = ptr
return reference{ptr}
} }
func (b *builder) PushAndChain(n Node) reference { func (b *builder) PushAndChain(n Node) reference {
newIdx := len(b.tree.nodes) ptr := b.push(n)
n.next = -1 if b.root.first == nil {
n.child = -1 b.root.first = ptr
b.tree.nodes = append(b.tree.nodes, n)
if b.lastIdx >= 0 {
b.tree.nodes[b.lastIdx].next = int32(newIdx) //nolint:gosec // TOML ASTs are small
} }
b.lastIdx = newIdx if b.last != nil {
return reference(b.lastIdx) b.last.next = ptr
}
b.last = ptr
return reference{ptr}
} }
func (b *builder) AttachChild(parent reference, child reference) { func (b *builder) AttachChild(parent reference, child reference) {
b.tree.nodes[parent].child = int32(child) //nolint:gosec // TOML ASTs are small parent.child = child.Node
} }
func (b *builder) Chain(from reference, to reference) { func (b *builder) Chain(from reference, to reference) {
b.tree.nodes[from].next = int32(to) //nolint:gosec // TOML ASTs are small from.next = to.Node
} }
+4 -16
View File
@@ -6,40 +6,28 @@ import "fmt"
type Kind int type Kind int
const ( const (
// Invalid represents an invalid meta node. // Meta
Invalid Kind = iota Invalid Kind = iota
// Comment represents a comment meta node.
Comment Comment
// Key represents a key meta node.
Key Key
// Table represents a top-level table. // Top level structures
Table Table
// ArrayTable represents a top-level array table.
ArrayTable ArrayTable
// KeyValue represents a top-level key value.
KeyValue KeyValue
// Array represents an array container value. // Containers values
Array Array
// InlineTable represents an inline table container value.
InlineTable InlineTable
// String represents a string value. // Values
String String
// Bool represents a boolean value.
Bool Bool
// Float represents a floating point value.
Float Float
// Integer represents an integer value.
Integer Integer
// LocalDate represents a a local date value.
LocalDate LocalDate
// LocalTime represents a local time value.
LocalTime LocalTime
// LocalDateTime represents a local date/time value.
LocalDateTime LocalDateTime
// DateTime represents a data/time value.
DateTime DateTime
) )
+40 -62
View File
@@ -5,7 +5,7 @@ import (
"fmt" "fmt"
"unicode" "unicode"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/characters" "github.com/pelletier/go-toml/v2/internal/characters"
) )
// ParserError describes an error relative to the content of the document. // ParserError describes an error relative to the content of the document.
@@ -69,26 +69,11 @@ func (p *Parser) Data() []byte {
// panics. // panics.
func (p *Parser) Range(b []byte) Range { func (p *Parser) Range(b []byte) Range {
return Range{ return Range{
Offset: uint32(p.subsliceOffset(b)), //nolint:gosec // TOML documents are small Offset: uint32(cap(p.data) - cap(b)),
Length: uint32(len(b)), //nolint:gosec // TOML documents are small Length: uint32(len(b)),
} }
} }
// rangeOfToken computes the Range of a token given the remaining bytes after the token.
// This is used when the token was extracted from the beginning of some position,
// and 'rest' is what remains after the token.
func (p *Parser) rangeOfToken(token, rest []byte) Range {
offset := len(p.data) - len(token) - len(rest)
return Range{Offset: uint32(offset), Length: uint32(len(token))} //nolint:gosec // TOML documents are small
}
// subsliceOffset returns the byte offset of subslice b within p.data.
// b must be a suffix (tail) of p.data.
func (p *Parser) subsliceOffset(b []byte) int {
// b is a suffix of p.data, so its offset is len(p.data) - len(b)
return len(p.data) - len(b)
}
// Raw returns the slice corresponding to the bytes in the given range. // Raw returns the slice corresponding to the bytes in the given range.
func (p *Parser) Raw(raw Range) []byte { func (p *Parser) Raw(raw Range) []byte {
return p.data[raw.Offset : raw.Offset+raw.Length] return p.data[raw.Offset : raw.Offset+raw.Length]
@@ -172,17 +157,9 @@ type Shape struct {
End Position End Position
} }
// Shape returns the shape of the given range in the input. Will func (p *Parser) position(b []byte) Position {
// panic if the range is not a subslice of the input. offset := cap(p.data) - cap(b)
func (p *Parser) Shape(r Range) Shape {
return Shape{
Start: p.positionAt(int(r.Offset)),
End: p.positionAt(int(r.Offset + r.Length)),
}
}
// positionAt returns the position at the given byte offset in the document.
func (p *Parser) positionAt(offset int) Position {
lead := p.data[:offset] lead := p.data[:offset]
return Position{ return Position{
@@ -192,6 +169,16 @@ func (p *Parser) positionAt(offset int) Position {
} }
} }
// Shape returns the shape of the given range in the input. Will
// panic if the range is not a subslice of the input.
func (p *Parser) Shape(r Range) Shape {
raw := p.Raw(r)
return Shape{
Start: p.position(raw),
End: p.position(raw[r.Length:]),
}
}
func (p *Parser) parseNewline(b []byte) ([]byte, error) { func (p *Parser) parseNewline(b []byte) ([]byte, error) {
if b[0] == '\n' { if b[0] == '\n' {
return b[1:], nil return b[1:], nil
@@ -211,7 +198,7 @@ func (p *Parser) parseComment(b []byte) (reference, []byte, error) {
if p.KeepComments && err == nil { if p.KeepComments && err == nil {
ref = p.builder.Push(Node{ ref = p.builder.Push(Node{
Kind: Comment, Kind: Comment,
Raw: p.rangeOfToken(data, rest), Raw: p.Range(data),
Data: data, Data: data,
}) })
} }
@@ -328,9 +315,6 @@ func (p *Parser) parseStdTable(b []byte) (reference, []byte, error) {
func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) { func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) {
// keyval = key keyval-sep val // keyval = key keyval-sep val
// Track the start position for Raw range
startB := b
ref := p.builder.Push(Node{ ref := p.builder.Push(Node{
Kind: KeyValue, Kind: KeyValue,
}) })
@@ -345,7 +329,7 @@ func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) {
b = p.parseWhitespace(b) b = p.parseWhitespace(b)
if len(b) == 0 { if len(b) == 0 {
return invalidReference, nil, NewParserError(startB[:len(startB)-len(b)], "expected = after a key, but the document ends there") return invalidReference, nil, NewParserError(b, "expected = after a key, but the document ends there")
} }
b, err = expect('=', b) b, err = expect('=', b)
@@ -363,11 +347,6 @@ func (p *Parser) parseKeyval(b []byte) (reference, []byte, error) {
p.builder.Chain(valRef, key) p.builder.Chain(valRef, key)
p.builder.AttachChild(ref, valRef) p.builder.AttachChild(ref, valRef)
// Set Raw to span the entire key-value expression.
// Access the node directly in the slice to avoid the write barrier
// that NodeAt's nodes-pointer setup would trigger.
p.builder.tree.nodes[ref].Raw = p.rangeOfToken(startB[:len(startB)-len(b)], b)
return ref, b, err return ref, b, err
} }
@@ -396,7 +375,7 @@ func (p *Parser) parseVal(b []byte) (reference, []byte, error) {
if err == nil { if err == nil {
ref = p.builder.Push(Node{ ref = p.builder.Push(Node{
Kind: String, Kind: String,
Raw: p.rangeOfToken(raw, b), Raw: p.Range(raw),
Data: v, Data: v,
}) })
} }
@@ -414,7 +393,7 @@ func (p *Parser) parseVal(b []byte) (reference, []byte, error) {
if err == nil { if err == nil {
ref = p.builder.Push(Node{ ref = p.builder.Push(Node{
Kind: String, Kind: String,
Raw: p.rangeOfToken(raw, b), Raw: p.Range(raw),
Data: v, Data: v,
}) })
} }
@@ -476,7 +455,7 @@ func (p *Parser) parseInlineTable(b []byte) (reference, []byte, error) {
// inline-table-keyvals = keyval [ inline-table-sep inline-table-keyvals ] // inline-table-keyvals = keyval [ inline-table-sep inline-table-keyvals ]
parent := p.builder.Push(Node{ parent := p.builder.Push(Node{
Kind: InlineTable, Kind: InlineTable,
Raw: p.rangeOfToken(b[:1], b[1:]), Raw: p.Range(b[:1]),
}) })
first := true first := true
@@ -562,7 +541,7 @@ func (p *Parser) parseValArray(b []byte) (reference, []byte, error) {
var err error var err error
for len(b) > 0 { for len(b) > 0 {
var cref reference cref := invalidReference
cref, b, err = p.parseOptionalWhitespaceCommentNewline(b) cref, b, err = p.parseOptionalWhitespaceCommentNewline(b)
if err != nil { if err != nil {
return parent, nil, err return parent, nil, err
@@ -631,13 +610,12 @@ func (p *Parser) parseOptionalWhitespaceCommentNewline(b []byte) (reference, []b
latestCommentRef := invalidReference latestCommentRef := invalidReference
addComment := func(ref reference) { addComment := func(ref reference) {
switch { if rootCommentRef == invalidReference {
case rootCommentRef == invalidReference:
rootCommentRef = ref rootCommentRef = ref
case latestCommentRef == invalidReference: } else if latestCommentRef == invalidReference {
p.builder.AttachChild(rootCommentRef, ref) p.builder.AttachChild(rootCommentRef, ref)
latestCommentRef = ref latestCommentRef = ref
default: } else {
p.builder.Chain(latestCommentRef, ref) p.builder.Chain(latestCommentRef, ref)
latestCommentRef = ref latestCommentRef = ref
} }
@@ -725,11 +703,11 @@ func (p *Parser) parseMultilineBasicString(b []byte) ([]byte, []byte, []byte, er
if !escaped { if !escaped {
str := token[startIdx:endIdx] str := token[startIdx:endIdx]
highlight := characters.Utf8TomlValidAlreadyEscaped(str) verr := characters.Utf8TomlValidAlreadyEscaped(str)
if len(highlight) == 0 { if verr.Zero() {
return token, str, rest, nil return token, str, rest, nil
} }
return nil, nil, nil, NewParserError(highlight, "invalid UTF-8") return nil, nil, nil, NewParserError(str[verr.Index:verr.Index+verr.Size], "invalid UTF-8")
} }
var builder bytes.Buffer var builder bytes.Buffer
@@ -765,7 +743,7 @@ func (p *Parser) parseMultilineBasicString(b []byte) ([]byte, []byte, []byte, er
i += j i += j
for ; i < len(token)-3; i++ { for ; i < len(token)-3; i++ {
c := token[i] c := token[i]
if c != '\n' && c != '\r' && c != ' ' && c != '\t' { if !(c == '\n' || c == '\r' || c == ' ' || c == '\t') {
i-- i--
break break
} }
@@ -841,7 +819,7 @@ func (p *Parser) parseKey(b []byte) (reference, []byte, error) {
ref := p.builder.Push(Node{ ref := p.builder.Push(Node{
Kind: Key, Kind: Key,
Raw: p.rangeOfToken(raw, b), Raw: p.Range(raw),
Data: key, Data: key,
}) })
@@ -857,7 +835,7 @@ func (p *Parser) parseKey(b []byte) (reference, []byte, error) {
p.builder.PushAndChain(Node{ p.builder.PushAndChain(Node{
Kind: Key, Kind: Key,
Raw: p.rangeOfToken(raw, b), Raw: p.Range(raw),
Data: key, Data: key,
}) })
} else { } else {
@@ -918,11 +896,11 @@ func (p *Parser) parseBasicString(b []byte) ([]byte, []byte, []byte, error) {
// validate the string and return a direct reference to the buffer. // validate the string and return a direct reference to the buffer.
if !escaped { if !escaped {
str := token[startIdx:endIdx] str := token[startIdx:endIdx]
highlight := characters.Utf8TomlValidAlreadyEscaped(str) verr := characters.Utf8TomlValidAlreadyEscaped(str)
if len(highlight) == 0 { if verr.Zero() {
return token, str, rest, nil return token, str, rest, nil
} }
return nil, nil, nil, NewParserError(highlight, "invalid UTF-8") return nil, nil, nil, NewParserError(str[verr.Index:verr.Index+verr.Size], "invalid UTF-8")
} }
i := startIdx i := startIdx
@@ -993,7 +971,7 @@ func hexToRune(b []byte, length int) (rune, error) {
var r uint32 var r uint32
for i, c := range b { for i, c := range b {
var d uint32 d := uint32(0)
switch { switch {
case '0' <= c && c <= '9': case '0' <= c && c <= '9':
d = uint32(c - '0') d = uint32(c - '0')
@@ -1034,7 +1012,7 @@ func (p *Parser) parseIntOrFloatOrDateTime(b []byte) (reference, []byte, error)
return p.builder.Push(Node{ return p.builder.Push(Node{
Kind: Float, Kind: Float,
Data: b[:3], Data: b[:3],
Raw: p.rangeOfToken(b[:3], b[3:]), Raw: p.Range(b[:3]),
}), b[3:], nil }), b[3:], nil
case 'n': case 'n':
if !scanFollowsNan(b) { if !scanFollowsNan(b) {
@@ -1044,7 +1022,7 @@ func (p *Parser) parseIntOrFloatOrDateTime(b []byte) (reference, []byte, error)
return p.builder.Push(Node{ return p.builder.Push(Node{
Kind: Float, Kind: Float,
Data: b[:3], Data: b[:3],
Raw: p.rangeOfToken(b[:3], b[3:]), Raw: p.Range(b[:3]),
}), b[3:], nil }), b[3:], nil
case '+', '-': case '+', '-':
return p.scanIntOrFloat(b) return p.scanIntOrFloat(b)
@@ -1169,7 +1147,7 @@ func (p *Parser) scanIntOrFloat(b []byte) (reference, []byte, error) {
return p.builder.Push(Node{ return p.builder.Push(Node{
Kind: Integer, Kind: Integer,
Data: b[:i], Data: b[:i],
Raw: p.rangeOfToken(b[:i], b[i:]), Raw: p.Range(b[:i]),
}), b[i:], nil }), b[i:], nil
} }
@@ -1193,7 +1171,7 @@ func (p *Parser) scanIntOrFloat(b []byte) (reference, []byte, error) {
return p.builder.Push(Node{ return p.builder.Push(Node{
Kind: Float, Kind: Float,
Data: b[:i+3], Data: b[:i+3],
Raw: p.rangeOfToken(b[:i+3], b[i+3:]), Raw: p.Range(b[:i+3]),
}), b[i+3:], nil }), b[i+3:], nil
} }
@@ -1205,7 +1183,7 @@ func (p *Parser) scanIntOrFloat(b []byte) (reference, []byte, error) {
return p.builder.Push(Node{ return p.builder.Push(Node{
Kind: Float, Kind: Float,
Data: b[:i+3], Data: b[:i+3],
Raw: p.rangeOfToken(b[:i+3], b[i+3:]), Raw: p.Range(b[:i+3]),
}), b[i+3:], nil }), b[i+3:], nil
} }
@@ -1228,7 +1206,7 @@ func (p *Parser) scanIntOrFloat(b []byte) (reference, []byte, error) {
return p.builder.Push(Node{ return p.builder.Push(Node{
Kind: kind, Kind: kind,
Data: b[:i], Data: b[:i],
Raw: p.rangeOfToken(b[:i], b[i:]), Raw: p.Range(b[:i]),
}), b[i:], nil }), b[i:], nil
} }
+10 -78
View File
@@ -6,7 +6,7 @@ import (
"strings" "strings"
"testing" "testing"
"git.ostiwe.com/ostiwe/go-toml/v2/internal/assert" "github.com/pelletier/go-toml/v2/internal/assert"
) )
func TestParser_AST_Numbers(t *testing.T) { func TestParser_AST_Numbers(t *testing.T) {
@@ -358,7 +358,7 @@ func BenchmarkParseBasicStringWithUnicode(b *testing.B) {
b.SetBytes(int64(len(input))) b.SetBytes(int64(len(input)))
for i := 0; i < b.N; i++ { for i := 0; i < b.N; i++ {
_, _, _, _ = p.parseBasicString(input) p.parseBasicString(input)
} }
}) })
b.Run("8", func(b *testing.B) { b.Run("8", func(b *testing.B) {
@@ -367,7 +367,7 @@ func BenchmarkParseBasicStringWithUnicode(b *testing.B) {
b.SetBytes(int64(len(input))) b.SetBytes(int64(len(input)))
for i := 0; i < b.N; i++ { for i := 0; i < b.N; i++ {
_, _, _, _ = p.parseBasicString(input) p.parseBasicString(input)
} }
}) })
} }
@@ -383,7 +383,7 @@ func BenchmarkParseBasicStringsEasy(b *testing.B) {
b.SetBytes(int64(len(input))) b.SetBytes(int64(len(input)))
for i := 0; i < b.N; i++ { for i := 0; i < b.N; i++ {
_, _, _, _ = p.parseBasicString(input) p.parseBasicString(input)
} }
}) })
} }
@@ -539,7 +539,7 @@ key5 = [ # Next to start of inline array.
// --- // ---
// 6:1->6:22 (105->126) | Comment [# Above simple value.] // 6:1->6:22 (105->126) | Comment [# Above simple value.]
// --- // ---
// 7:1->7:14 (127->140) | KeyValue [] // 1:1->1:1 (0->0) | KeyValue []
// 7:7->7:14 (133->140) | String [value] // 7:7->7:14 (133->140) | String [value]
// 7:1->7:4 (127->130) | Key [key] // 7:1->7:4 (127->130) | Key [key]
// 7:15->7:38 (141->164) | Comment [# Next to simple value.] // 7:15->7:38 (141->164) | Comment [# Next to simple value.]
@@ -552,12 +552,12 @@ key5 = [ # Next to start of inline array.
// --- // ---
// 14:1->14:22 (252->273) | Comment [# Above inline table.] // 14:1->14:22 (252->273) | Comment [# Above inline table.]
// --- // ---
// 15:1->15:50 (274->323) | KeyValue [] // 1:1->1:1 (0->0) | KeyValue []
// 15:8->15:9 (281->282) | InlineTable [] // 15:8->15:9 (281->282) | InlineTable []
// 15:10->15:23 (283->296) | KeyValue [] // 1:1->1:1 (0->0) | KeyValue []
// 15:18->15:23 (291->296) | String [Tom] // 15:18->15:23 (291->296) | String [Tom]
// 15:10->15:15 (283->288) | Key [first] // 15:10->15:15 (283->288) | Key [first]
// 15:25->15:48 (298->321) | KeyValue [] // 1:1->1:1 (0->0) | KeyValue []
// 15:32->15:48 (305->321) | String [Preston-Werner] // 15:32->15:48 (305->321) | String [Preston-Werner]
// 15:25->15:29 (298->302) | Key [last] // 15:25->15:29 (298->302) | Key [last]
// 15:1->15:5 (274->278) | Key [name] // 15:1->15:5 (274->278) | Key [name]
@@ -567,7 +567,7 @@ key5 = [ # Next to start of inline array.
// --- // ---
// 18:1->18:15 (371->385) | Comment [# Above array.] // 18:1->18:15 (371->385) | Comment [# Above array.]
// --- // ---
// 19:1->19:20 (386->405) | KeyValue [] // 1:1->1:1 (0->0) | KeyValue []
// 1:1->1:1 (0->0) | Array [] // 1:1->1:1 (0->0) | Array []
// 19:11->19:12 (396->397) | Integer [1] // 19:11->19:12 (396->397) | Integer [1]
// 19:14->19:15 (399->400) | Integer [2] // 19:14->19:15 (399->400) | Integer [2]
@@ -579,7 +579,7 @@ key5 = [ # Next to start of inline array.
// --- // ---
// 22:1->22:26 (448->473) | Comment [# Above multi-line array.] // 22:1->22:26 (448->473) | Comment [# Above multi-line array.]
// --- // ---
// 23:1->31:2 (474->694) | KeyValue [] // 1:1->1:1 (0->0) | KeyValue []
// 1:1->1:1 (0->0) | Array [] // 1:1->1:1 (0->0) | Array []
// 23:10->23:42 (483->515) | Comment [# Next to start of inline array.] // 23:10->23:42 (483->515) | Comment [# Next to start of inline array.]
// 24:3->24:38 (518->553) | Comment [# Second line before array content.] // 24:3->24:38 (518->553) | Comment [# Second line before array content.]
@@ -605,74 +605,6 @@ key5 = [ # Next to start of inline array.
// 36:1->36:21 (804->824) | Comment [# After array table.] // 36:1->36:21 (804->824) | Comment [# After array table.]
} }
func TestIterator_IsLast(t *testing.T) {
// Test IsLast on an iterator with multiple elements using public Parser API
doc := `array = [1, 2, 3]`
p := Parser{}
p.Reset([]byte(doc))
p.NextExpression()
e := p.Expression()
arr := e.Value() // The array node
it := arr.Children()
count := 0
lastCount := 0
for it.Next() {
count++
if it.IsLast() {
lastCount++
}
}
assert.Equal(t, 3, count)
assert.Equal(t, 1, lastCount)
}
func TestNodeChaining(t *testing.T) {
// Test that sibling nodes are correctly chained via Next()
// This exercises the internal PushAndChain functionality through public APIs
doc := `a.b.c = 1`
p := Parser{}
p.Reset([]byte(doc))
p.NextExpression()
e := p.Expression()
// KeyValue has children: value, then key parts (a, b, c)
keyIt := e.Key()
// Collect all key parts by following the iterator
var keys []string
for keyIt.Next() {
keys = append(keys, string(keyIt.Node().Data))
}
assert.Equal(t, []string{"a", "b", "c"}, keys)
}
func TestMultipleExpressions(t *testing.T) {
// Test parsing multiple top-level expressions
// This exercises root iteration through public APIs
doc := `
key1 = "value1"
key2 = "value2"
key3 = "value3"
`
p := Parser{}
p.Reset([]byte(doc))
var keys []string
for p.NextExpression() {
e := p.Expression()
keyIt := e.Key()
keyIt.Next()
keys = append(keys, string(keyIt.Node().Data))
}
assert.NoError(t, p.Error())
assert.Equal(t, []string{"key1", "key2", "key3"}, keys)
}
func ExampleParser() { func ExampleParser() {
doc := ` doc := `
hello = "world" hello = "world"
+1 -1
View File
@@ -1,6 +1,6 @@
package unstable package unstable
import "git.ostiwe.com/ostiwe/go-toml/v2/internal/characters" import "github.com/pelletier/go-toml/v2/internal/characters"
func scanFollows(b []byte, pattern string) bool { func scanFollows(b []byte, pattern string) bool {
n := len(pattern) n := len(pattern)
+3 -28
View File
@@ -1,32 +1,7 @@
package unstable package unstable
// Unmarshaler is implemented by types that can unmarshal a TOML // The Unmarshaler interface may be implemented by types to customize their
// description of themselves. The input is a valid TOML document // behavior when being unmarshaled from a TOML document.
// containing the relevant portion of the parsed document.
//
// For tables (including split tables defined in multiple places),
// the data contains the raw key-value bytes from the original document
// with adjusted table headers to be relative to the unmarshaling target.
type Unmarshaler interface { type Unmarshaler interface {
UnmarshalTOML(data []byte) error UnmarshalTOML(value *Node) error
}
// RawMessage is a raw encoded TOML value. It implements Unmarshaler
// and can be used to delay TOML decoding or capture raw content.
//
// Example usage:
//
// type Config struct {
// Plugin RawMessage `toml:"plugin"`
// }
//
// var cfg Config
// toml.NewDecoder(r).EnableUnmarshalerInterface().Decode(&cfg)
// // cfg.Plugin now contains the raw TOML bytes for [plugin]
type RawMessage []byte
// UnmarshalTOML implements Unmarshaler.
func (m *RawMessage) UnmarshalTOML(data []byte) error {
*m = append((*m)[0:0], data...)
return nil
} }