9b890cf9c5
go.mod: bump minimum and language to 1.21 ( #949 )
...
* go.mod: bump minimum and language to 1.21
CI only tests Go 1.21 and 1.22, and older versions of Go are no longer
getting any bug or security fixes, so advertise that we only support
Go 1.21 or later via go.mod.
While here, ensure the module is tidy and resolve deprecation warnings,
and remove now-unnecessary Go version build tags.
* replace sort.Slice with slices.SortFunc
The latter is more efficient, and allocates less, since sort.Slice
needs to go through sort.Interface which causes allocations.
goos: linux
goarch: amd64
pkg: github.com/pelletier/go-toml/v2/benchmark
cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
│ old │ new │
│ sec/op │ sec/op vs base │
Marshal/HugoFrontMatter-8 7.612µ ± 1% 6.730µ ± 1% -11.59% (p=0.002 n=6)
│ old │ new │
│ B/s │ B/s vs base │
Marshal/HugoFrontMatter-8 65.52Mi ± 1% 74.11Mi ± 1% +13.11% (p=0.002 n=6)
│ old │ new │
│ B/op │ B/op vs base │
Marshal/HugoFrontMatter-8 5.672Ki ± 0% 5.266Ki ± 0% -7.16% (p=0.002 n=6)
│ old │ new │
│ allocs/op │ allocs/op vs base │
Marshal/HugoFrontMatter-8 85.00 ± 0% 73.00 ± 0% -14.12% (p=0.002 n=6)
2024-05-24 10:58:39 -04:00
8ed6d131eb
Decode: unstable/Unmarshal interface ( #940 )
...
Co-authored-by: Pavlos Karakalidis <pkarakal@pkarakal.com >
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes >
2024-03-19 12:33:12 -04:00
06fb30bf2e
Decode: fix reuse of slice for array tables ( #934 )
...
When decoding into a non-empty slice, it needs to be emptied so that only the
tables contained in the document are present in the resulting value.
Arrays are not impacted because their unmarshal offset is tracked separately.
Fixes #931
2024-02-27 15:28:49 -05:00
34765b4a9e
Fix unmarshaling of nested non-exported struct ( #917 )
...
Fixes #915
2023-12-11 14:17:49 -05:00
4835627845
Decode: improve errors on integers and strings ( #891 )
2023-08-28 11:17:48 -04:00
e183db7e69
Decode: assigned empty struct to empty defined sections ( #879 )
...
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes >
2023-07-12 10:53:17 -04:00
7b980e792b
Use PtrTo to not require Go 1.18 ( #874 )
2023-05-23 18:22:22 -04:00
986afffb7c
Decode: fix decode into unsettable structs ( #868 )
...
Fixes #866
2023-05-16 09:29:50 -04:00
d34104d493
Support text Un/Marshaller for map keys ( #863 )
2023-05-09 17:56:57 +02:00
643c251c4b
Fix panic when unmarshaling into a map twice ( #854 )
...
Fixes #851
2023-02-28 17:34:24 +01:00
8a416daa69
Fix error report of type mismatch on inline tables ( #853 )
...
Parser did not track the location of the faulty inline table in the
document, and unmarshaler tried to the use the non-raw data field of the
AST node, both resulting into a panic when generating the parser error.
Fixes #850
2023-02-28 17:06:49 +01:00
fcd9179b7d
Fixes typos ( #849 )
2023-02-13 03:57:48 -08:00
9f5726004e
Allow integers to be unmarshaled into floats ( #841 )
...
Co-authored-by: Marty <martin@windscribe.com >
2023-02-09 12:02:25 -05:00
e195b58fd0
Expose parser API as unstable ( #827 )
2022-11-09 16:12:39 -05:00
b9e3b9c370
refactor: Use typeMismatchError rather than raw string error ( #826 )
...
Uses the existing method to DRY up the error message generation, and decorates
with position index where needed. No behaviour is changed, but it allows for
further changes to make error messaging more specific.
Related to: pelletier/go-toml#806
2022-10-30 13:44:16 -04:00
942841787a
Fix reflect.Pointer backward compatibility ( #813 )
...
Though we don't officially support older versions of Go, this is an easy fix to
unblock people.
Fixes #812
2022-08-26 09:15:03 -04:00
28f1efc7d3
Decode: don't break on non-struct embed field ( #810 )
2022-08-22 18:39:11 -04:00
e46d245c09
Decode: don't crash on embedded nil pointers ( #808 )
...
Also has the perks of reducing the overhead of FindByIndex:
```
name old time/op new time/op delta
UnmarshalDataset/config-32 17.0ms ± 1% 17.0ms ± 1% ~ (p=1.000 n=5+5)
UnmarshalDataset/canada-32 71.6ms ± 1% 71.4ms ± 1% ~ (p=1.000 n=5+5)
UnmarshalDataset/citm_catalog-32 24.2ms ± 3% 23.5ms ± 2% -3.03% (p=0.032 n=5+5)
UnmarshalDataset/twitter-32 9.37ms ± 1% 9.09ms ± 2% -2.97% (p=0.032 n=5+5)
UnmarshalDataset/code-32 75.4ms ± 2% 74.9ms ± 0% ~ (p=0.222 n=5+5)
UnmarshalDataset/example-32 147µs ±10% 136µs ± 1% -7.14% (p=0.008 n=5+5)
Unmarshal/SimpleDocument/struct-32 512ns ± 2% 500ns ± 0% -2.35% (p=0.008 n=5+5)
Unmarshal/SimpleDocument/map-32 721ns ± 2% 702ns ± 1% -2.68% (p=0.008 n=5+5)
Unmarshal/ReferenceFile/struct-32 40.1µs ± 0% 39.6µs ± 0% -1.30% (p=0.008 n=5+5)
Unmarshal/ReferenceFile/map-32 62.3µs ± 1% 60.6µs ± 0% -2.83% (p=0.008 n=5+5)
Unmarshal/HugoFrontMatter-32 10.8µs ± 1% 10.5µs ± 1% -2.86% (p=0.008 n=5+5)
name old speed new speed delta
UnmarshalDataset/config-32 61.8MB/s ± 1% 61.8MB/s ± 1% ~ (p=1.000 n=5+5)
UnmarshalDataset/canada-32 30.8MB/s ± 1% 30.8MB/s ± 1% ~ (p=1.000 n=5+5)
UnmarshalDataset/citm_catalog-32 23.0MB/s ± 3% 23.8MB/s ± 2% +3.09% (p=0.032 n=5+5)
UnmarshalDataset/twitter-32 47.2MB/s ± 1% 48.6MB/s ± 2% +3.09% (p=0.032 n=5+5)
UnmarshalDataset/code-32 35.6MB/s ± 2% 35.9MB/s ± 0% ~ (p=0.222 n=5+5)
UnmarshalDataset/example-32 55.3MB/s ±10% 59.4MB/s ± 1% +7.36% (p=0.008 n=5+5)
Unmarshal/SimpleDocument/struct-32 21.5MB/s ± 2% 22.0MB/s ± 0% +2.41% (p=0.008 n=5+5)
Unmarshal/SimpleDocument/map-32 15.2MB/s ± 2% 15.7MB/s ± 1% +2.74% (p=0.008 n=5+5)
Unmarshal/ReferenceFile/struct-32 131MB/s ± 0% 132MB/s ± 0% +1.31% (p=0.008 n=5+5)
Unmarshal/ReferenceFile/map-32 84.1MB/s ± 1% 86.6MB/s ± 0% +2.91% (p=0.008 n=5+5)
Unmarshal/HugoFrontMatter-32 50.6MB/s ± 1% 52.1MB/s ± 1% +2.93% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
UnmarshalDataset/config-32 5.86MB ± 0% 5.86MB ± 0% ~ (p=0.579 n=5+5)
UnmarshalDataset/canada-32 83.0MB ± 0% 83.0MB ± 0% ~ (p=0.651 n=5+5)
UnmarshalDataset/citm_catalog-32 34.7MB ± 0% 34.7MB ± 0% ~ (p=0.548 n=5+5)
UnmarshalDataset/twitter-32 12.7MB ± 0% 12.7MB ± 0% ~ (p=1.000 n=5+5)
UnmarshalDataset/code-32 22.2MB ± 0% 22.2MB ± 0% ~ (p=0.841 n=5+5)
UnmarshalDataset/example-32 186kB ± 0% 186kB ± 0% ~ (p=0.111 n=5+5)
Unmarshal/SimpleDocument/struct-32 805B ± 0% 805B ± 0% ~ (all equal)
Unmarshal/SimpleDocument/map-32 1.13kB ± 0% 1.13kB ± 0% ~ (all equal)
Unmarshal/ReferenceFile/struct-32 20.9kB ± 0% 20.9kB ± 0% ~ (p=0.643 n=5+5)
Unmarshal/ReferenceFile/map-32 38.3kB ± 0% 38.3kB ± 0% ~ (p=0.397 n=5+5)
Unmarshal/HugoFrontMatter-32 7.44kB ± 0% 7.44kB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
UnmarshalDataset/config-32 227k ± 0% 227k ± 0% ~ (p=1.000 n=5+5)
UnmarshalDataset/canada-32 782k ± 0% 782k ± 0% ~ (all equal)
UnmarshalDataset/citm_catalog-32 192k ± 0% 192k ± 0% ~ (p=0.968 n=4+5)
UnmarshalDataset/twitter-32 56.9k ± 0% 56.9k ± 0% ~ (p=0.429 n=4+5)
UnmarshalDataset/code-32 1.05M ± 0% 1.05M ± 0% ~ (p=0.556 n=4+5)
UnmarshalDataset/example-32 1.36k ± 0% 1.36k ± 0% ~ (all equal)
Unmarshal/SimpleDocument/struct-32 9.00 ± 0% 9.00 ± 0% ~ (all equal)
Unmarshal/SimpleDocument/map-32 13.0 ± 0% 13.0 ± 0% ~ (all equal)
Unmarshal/ReferenceFile/struct-32 183 ± 0% 183 ± 0% ~ (all equal)
Unmarshal/ReferenceFile/map-32 642 ± 0% 642 ± 0% ~ (all equal)
Unmarshal/HugoFrontMatter-32 141 ± 0% 141 ± 0% ~ (all equal)
```
Fixes #807
2022-08-20 21:24:03 -04:00
7baa23f493
Decode: error on array table mismatched type ( #804 )
...
Prevent the decoder from continuing if it encounters a type it cannot decode an
array table into.
Fixes #799
2022-08-15 16:38:07 -04:00
67bc5422f3
Go 1.19 ( #802 )
2022-08-15 10:56:33 -04:00
0a422e3dbd
Decoder: check max uint on 32 bit platforms ( #778 )
...
Fixes #777
2022-05-10 15:43:26 +02:00
c5ca2c682b
Fix embedded struct with explicit field name ( #773 )
...
Fixes #772
2022-05-09 18:45:02 +02:00
8bbb673431
Fuzzing setup and fixes ( #755 )
...
* encode: fix localdate formatting
* encode: fix empty key marshaling
* encode: fix invalid quotation of time.Time
* encode: ensure control chars are escaped
* decode: always use UTC for zero tz
* encode: check for invalid characters in keys
* encode: always construct map for empty array tables
* fuzz: add go 1.18 fuzz test
* encode: handle NaNs
* encode: allow new lines in quoted keys
* encode: never emit table inside array
* encode: don't capitalize inf
2022-04-10 21:37:12 -04:00
f5cc8c49eb
decoder: remove mention of UnmarshalText in errors ( #751 )
...
Fixes #737
2022-04-07 21:58:19 -04:00
3229a0abfb
Decode: convert table key to correct type ( #741 )
...
Fixes #740 .
2022-03-02 09:24:01 -05:00
e83cf535f5
Decoder: rename SetStrict to DisallowUnknownFields ( #731 )
2022-01-02 14:32:34 -05:00
d58eb50ebf
Doc: clarify errors returned by Decode ( #713 )
...
Fixes #625
2021-12-26 20:04:09 +01:00
8ce5c3d78f
Decoder: time allows extra precision ( #710 )
...
As discussed[1], this change allows times to provide precision beyond the
nanosecond (nine digits fractional part). Extra precision is truncated according
to the TOML specificiation.
[1]: https://github.com/pelletier/go-toml/discussions/707
2021-12-26 17:05:10 +01:00
c862c344b3
Decoder: allow commas in tags ( #693 )
2021-11-30 21:59:22 -05:00
b226db6a29
Decoder: show struct field in type mismatch errors ( #684 )
...
The goal is to provide some context as to why the type were mismatched. This
change only works for that case, on structs. This is the same a encoding/json. A
more general solution would be great, but this would require a broader change in
the decoder, which I don't think is necessary at the moment.
Fixes #628
2021-11-24 20:43:56 -05:00
79e78b234c
Decoder: fix panic on table array behind a pointer ( #682 )
...
Fixes #677
2021-11-24 18:50:04 -05:00
64fe47161f
API: Encoder and Decoder options are chainable ( #670 )
...
Fixes #583
2021-11-13 19:04:53 -05:00
dc1740d473
Decode: code cleanup for struct cache ( #659 )
2021-11-07 18:35:30 -05:00
c871a61015
unmarshal: use UnmarshalText for any type ( #642 )
...
Not only structs can implement TextUnmarshaler.
Fixes #564
2021-10-28 17:02:47 -04:00
d0d001625c
unmarshal: don't panic when storing table in slice ( #641 )
...
New error message:
```
toml: cannot store a table in a slice
1| [things]
| ~~~~~~ cannot store a table in a slice
2| foo = "bar"
```
Fixes #623
2021-10-25 16:47:10 -04:00
64941b99e2
unmarshal: empty document results in map ( #640 )
...
Fixes #602
2021-10-25 15:55:54 -04:00
ed02a1f192
seen: check for explicit tables on dotted keys ( #639 )
...
The TOML spec is being clarified to say that dotted keys "define" their
intermediate tables. Therefore the seen tracker needs to verify that none of
them reference an explicit table.
Also added a missing seen expression check for key-values parsed as part of a
table section.
See https://github.com/toml-lang/toml/issues/846
2021-10-22 23:25:28 -04:00
cd54472d03
Validate UTF-8 ( #629 )
2021-10-15 19:13:21 -04:00
ee9b902222
unmarshal: convert ints if target type is compatible ( #594 )
...
This is required to support custom types.
Fixes #590
2021-09-09 21:25:14 -04:00
1230ca485e
unmarshal: make copy of non addressable values ( #576 )
...
When unmarshaling into a nested struct in a map, the value is not
addressable. In that case, make a copy of it and modify it instead.
Fixes #575
2021-08-31 20:22:38 -04:00
8be357dfa1
Add LocalTime to interface{} decode support ( #567 )
...
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes >
2021-07-21 17:50:12 +02:00
f6b38c33b7
Provide own implementation of Local* ( #558 )
...
* Reduces the public API.
* Reuses optimized parsing functions.
* Removes reliance on Google code under Apache license.
2021-06-08 20:27:05 -04:00
773f10110c
Unmarshal recursive structs ( #557 )
...
Co-authored-by: Nabetani <takenori@nabetani.sakura.ne.jp >
2021-06-08 14:22:39 -04:00
618f0181ac
AST Tweaks ( #551 )
...
* Use pointers instead of copying around ast.Node
Node is a 56B struct that is constantly in the hot path. Passing nodes
around by copy had a cost that started to add up. This change replaces
them by pointers. Using unsafe pointer arithmetic and converting
sibling/child indexes to relative offsets, it removes the need to carry
around a pointer to the root of the tree. This saves 8B per Node. This
space will be used to store an extra []byte slice to provide contextual
error handling on all nodes, including the ones whose data is different
than the raw input (for example: strings with escaped characters), while
staying under the size of a cache line.
* Remove conditional
* Add Raw to track range in data for parsed values
* Simplify reference tracking
2021-06-03 21:48:51 -04:00
f3bb20ea79
Benchmark marshal ( #550 )
2021-06-02 09:29:19 -04:00
250e073408
Stack-based unmarshaler ( #546 )
...
* Benchmark script
* Rewrite unmarshaler using the stack
Instead of tracking the build chain using `target`s, use the stack
instead.
Working and most benchmarks look good, but regression on structs unmarshalling.
~60% slower on ReferenceFile/struct.
* Shortcut to check if last node of iterator
* Remove unecessary pointer allocation
* Skip over unused keys without marking them as seen
* Add some tests
* Fix mktemp on macos
2021-05-31 12:14:13 -04:00
11f022ab09
Fix parser skipping over the whole file ( #547 )
...
* Add test for ReferenceFile/struct
* Stop skipping after table
* Add .gitattributes to force LF encoding
* Fix the reference file
2021-05-30 18:53:07 -04:00
95c701b253
Increase test coverage ( #538 )
...
Also fix array in map bug.
2021-05-10 20:17:05 -04:00
45ea20024b
Readme ( #535 )
2021-05-08 17:03:51 -04:00
ea225df3ed
v2: errors ( #534 )
...
```
name old time/op new time/op delta
UnmarshalDataset/config-32 86.7ms ± 2% 87.5ms ± 2% ~ (p=0.113 n=9+10)
UnmarshalDataset/canada-32 129ms ± 4% 106ms ± 3% -17.94% (p=0.000 n=10+10)
UnmarshalDataset/citm_catalog-32 59.4ms ± 5% 58.7ms ± 5% ~ (p=0.393 n=10+10)
UnmarshalDataset/twitter-32 27.0ms ± 7% 26.9ms ± 6% ~ (p=0.720 n=10+9)
UnmarshalDataset/code-32 326ms ± 4% 322ms ± 7% ~ (p=0.661 n=9+10)
UnmarshalDataset/example-32 510µs ±11% 526µs ± 7% ~ (p=0.182 n=10+9)
UnmarshalSimple-32 1.41µs ± 6% 1.41µs ± 4% ~ (p=0.736 n=10+9)
ReferenceFile-32 45.6µs ± 3% 43.9µs ±10% ~ (p=0.089 n=10+10)
name old speed new speed delta
UnmarshalDataset/config-32 12.1MB/s ± 2% 12.0MB/s ± 2% ~ (p=0.108 n=9+10)
UnmarshalDataset/canada-32 17.1MB/s ± 4% 20.9MB/s ± 3% +21.86% (p=0.000 n=10+10)
UnmarshalDataset/citm_catalog-32 9.41MB/s ± 5% 9.51MB/s ± 5% ~ (p=0.362 n=10+10)
UnmarshalDataset/twitter-32 16.4MB/s ± 8% 16.5MB/s ± 6% ~ (p=0.704 n=10+9)
UnmarshalDataset/code-32 8.24MB/s ± 4% 8.34MB/s ± 7% ~ (p=0.675 n=9+10)
UnmarshalDataset/example-32 15.9MB/s ±11% 15.4MB/s ± 7% ~ (p=0.182 n=10+9)
ReferenceFile-32 115MB/s ± 4% 120MB/s ±10% ~ (p=0.085 n=10+10)
name old alloc/op new alloc/op delta
UnmarshalDataset/config-32 16.9MB ± 0% 16.9MB ± 0% -0.02% (p=0.000 n=10+10)
UnmarshalDataset/canada-32 76.8MB ± 0% 74.3MB ± 0% -3.31% (p=0.000 n=10+10)
UnmarshalDataset/citm_catalog-32 37.3MB ± 0% 37.1MB ± 0% -0.60% (p=0.000 n=9+10)
UnmarshalDataset/twitter-32 15.6MB ± 0% 15.6MB ± 0% -0.09% (p=0.000 n=10+10)
UnmarshalDataset/code-32 60.2MB ± 0% 59.3MB ± 0% -1.51% (p=0.000 n=10+9)
UnmarshalDataset/example-32 238kB ± 0% 238kB ± 0% -0.18% (p=0.000 n=10+10)
ReferenceFile-32 11.8kB ± 0% 11.8kB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
UnmarshalDataset/config-32 653k ± 0% 645k ± 0% -1.20% (p=0.000 n=10+6)
UnmarshalDataset/canada-32 1.01M ± 0% 0.90M ± 0% -11.04% (p=0.000 n=9+10)
UnmarshalDataset/citm_catalog-32 384k ± 0% 370k ± 0% -3.75% (p=0.000 n=10+10)
UnmarshalDataset/twitter-32 160k ± 0% 157k ± 0% -1.32% (p=0.000 n=10+10)
UnmarshalDataset/code-32 2.97M ± 0% 2.91M ± 0% -2.15% (p=0.000 n=10+7)
UnmarshalDataset/example-32 3.69k ± 0% 3.63k ± 0% -1.52% (p=0.000 n=10+10)
ReferenceFile-32 253 ± 0% 253 ± 0% ~ (all equal)
```
2021-05-08 16:04:25 -04:00
Daniel Martí