28f1efc7d3
Decode: don't break on non-struct embed field ( #810 )
2022-08-22 18:39:11 -04:00
e46d245c09
Decode: don't crash on embedded nil pointers ( #808 )
...
Also has the perks of reducing the overhead of FindByIndex:
```
name old time/op new time/op delta
UnmarshalDataset/config-32 17.0ms ± 1% 17.0ms ± 1% ~ (p=1.000 n=5+5)
UnmarshalDataset/canada-32 71.6ms ± 1% 71.4ms ± 1% ~ (p=1.000 n=5+5)
UnmarshalDataset/citm_catalog-32 24.2ms ± 3% 23.5ms ± 2% -3.03% (p=0.032 n=5+5)
UnmarshalDataset/twitter-32 9.37ms ± 1% 9.09ms ± 2% -2.97% (p=0.032 n=5+5)
UnmarshalDataset/code-32 75.4ms ± 2% 74.9ms ± 0% ~ (p=0.222 n=5+5)
UnmarshalDataset/example-32 147µs ±10% 136µs ± 1% -7.14% (p=0.008 n=5+5)
Unmarshal/SimpleDocument/struct-32 512ns ± 2% 500ns ± 0% -2.35% (p=0.008 n=5+5)
Unmarshal/SimpleDocument/map-32 721ns ± 2% 702ns ± 1% -2.68% (p=0.008 n=5+5)
Unmarshal/ReferenceFile/struct-32 40.1µs ± 0% 39.6µs ± 0% -1.30% (p=0.008 n=5+5)
Unmarshal/ReferenceFile/map-32 62.3µs ± 1% 60.6µs ± 0% -2.83% (p=0.008 n=5+5)
Unmarshal/HugoFrontMatter-32 10.8µs ± 1% 10.5µs ± 1% -2.86% (p=0.008 n=5+5)
name old speed new speed delta
UnmarshalDataset/config-32 61.8MB/s ± 1% 61.8MB/s ± 1% ~ (p=1.000 n=5+5)
UnmarshalDataset/canada-32 30.8MB/s ± 1% 30.8MB/s ± 1% ~ (p=1.000 n=5+5)
UnmarshalDataset/citm_catalog-32 23.0MB/s ± 3% 23.8MB/s ± 2% +3.09% (p=0.032 n=5+5)
UnmarshalDataset/twitter-32 47.2MB/s ± 1% 48.6MB/s ± 2% +3.09% (p=0.032 n=5+5)
UnmarshalDataset/code-32 35.6MB/s ± 2% 35.9MB/s ± 0% ~ (p=0.222 n=5+5)
UnmarshalDataset/example-32 55.3MB/s ±10% 59.4MB/s ± 1% +7.36% (p=0.008 n=5+5)
Unmarshal/SimpleDocument/struct-32 21.5MB/s ± 2% 22.0MB/s ± 0% +2.41% (p=0.008 n=5+5)
Unmarshal/SimpleDocument/map-32 15.2MB/s ± 2% 15.7MB/s ± 1% +2.74% (p=0.008 n=5+5)
Unmarshal/ReferenceFile/struct-32 131MB/s ± 0% 132MB/s ± 0% +1.31% (p=0.008 n=5+5)
Unmarshal/ReferenceFile/map-32 84.1MB/s ± 1% 86.6MB/s ± 0% +2.91% (p=0.008 n=5+5)
Unmarshal/HugoFrontMatter-32 50.6MB/s ± 1% 52.1MB/s ± 1% +2.93% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
UnmarshalDataset/config-32 5.86MB ± 0% 5.86MB ± 0% ~ (p=0.579 n=5+5)
UnmarshalDataset/canada-32 83.0MB ± 0% 83.0MB ± 0% ~ (p=0.651 n=5+5)
UnmarshalDataset/citm_catalog-32 34.7MB ± 0% 34.7MB ± 0% ~ (p=0.548 n=5+5)
UnmarshalDataset/twitter-32 12.7MB ± 0% 12.7MB ± 0% ~ (p=1.000 n=5+5)
UnmarshalDataset/code-32 22.2MB ± 0% 22.2MB ± 0% ~ (p=0.841 n=5+5)
UnmarshalDataset/example-32 186kB ± 0% 186kB ± 0% ~ (p=0.111 n=5+5)
Unmarshal/SimpleDocument/struct-32 805B ± 0% 805B ± 0% ~ (all equal)
Unmarshal/SimpleDocument/map-32 1.13kB ± 0% 1.13kB ± 0% ~ (all equal)
Unmarshal/ReferenceFile/struct-32 20.9kB ± 0% 20.9kB ± 0% ~ (p=0.643 n=5+5)
Unmarshal/ReferenceFile/map-32 38.3kB ± 0% 38.3kB ± 0% ~ (p=0.397 n=5+5)
Unmarshal/HugoFrontMatter-32 7.44kB ± 0% 7.44kB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
UnmarshalDataset/config-32 227k ± 0% 227k ± 0% ~ (p=1.000 n=5+5)
UnmarshalDataset/canada-32 782k ± 0% 782k ± 0% ~ (all equal)
UnmarshalDataset/citm_catalog-32 192k ± 0% 192k ± 0% ~ (p=0.968 n=4+5)
UnmarshalDataset/twitter-32 56.9k ± 0% 56.9k ± 0% ~ (p=0.429 n=4+5)
UnmarshalDataset/code-32 1.05M ± 0% 1.05M ± 0% ~ (p=0.556 n=4+5)
UnmarshalDataset/example-32 1.36k ± 0% 1.36k ± 0% ~ (all equal)
Unmarshal/SimpleDocument/struct-32 9.00 ± 0% 9.00 ± 0% ~ (all equal)
Unmarshal/SimpleDocument/map-32 13.0 ± 0% 13.0 ± 0% ~ (all equal)
Unmarshal/ReferenceFile/struct-32 183 ± 0% 183 ± 0% ~ (all equal)
Unmarshal/ReferenceFile/map-32 642 ± 0% 642 ± 0% ~ (all equal)
Unmarshal/HugoFrontMatter-32 141 ± 0% 141 ± 0% ~ (all equal)
```
Fixes #807
2022-08-20 21:24:03 -04:00
7baa23f493
Decode: error on array table mismatched type ( #804 )
...
Prevent the decoder from continuing if it encounters a type it cannot decode an
array table into.
Fixes #799
2022-08-15 16:38:07 -04:00
67bc5422f3
Go 1.19 ( #802 )
2022-08-15 10:56:33 -04:00
0a422e3dbd
Decoder: check max uint on 32 bit platforms ( #778 )
...
Fixes #777
2022-05-10 15:43:26 +02:00
c5ca2c682b
Fix embedded struct with explicit field name ( #773 )
...
Fixes #772
2022-05-09 18:45:02 +02:00
8bbb673431
Fuzzing setup and fixes ( #755 )
...
* encode: fix localdate formatting
* encode: fix empty key marshaling
* encode: fix invalid quotation of time.Time
* encode: ensure control chars are escaped
* decode: always use UTC for zero tz
* encode: check for invalid characters in keys
* encode: always construct map for empty array tables
* fuzz: add go 1.18 fuzz test
* encode: handle NaNs
* encode: allow new lines in quoted keys
* encode: never emit table inside array
* encode: don't capitalize inf
2022-04-10 21:37:12 -04:00
f5cc8c49eb
decoder: remove mention of UnmarshalText in errors ( #751 )
...
Fixes #737
2022-04-07 21:58:19 -04:00
3229a0abfb
Decode: convert table key to correct type ( #741 )
...
Fixes #740 .
2022-03-02 09:24:01 -05:00
e83cf535f5
Decoder: rename SetStrict to DisallowUnknownFields ( #731 )
2022-01-02 14:32:34 -05:00
d58eb50ebf
Doc: clarify errors returned by Decode ( #713 )
...
Fixes #625
2021-12-26 20:04:09 +01:00
8ce5c3d78f
Decoder: time allows extra precision ( #710 )
...
As discussed[1], this change allows times to provide precision beyond the
nanosecond (nine digits fractional part). Extra precision is truncated according
to the TOML specificiation.
[1]: https://github.com/pelletier/go-toml/discussions/707
2021-12-26 17:05:10 +01:00
c862c344b3
Decoder: allow commas in tags ( #693 )
2021-11-30 21:59:22 -05:00
b226db6a29
Decoder: show struct field in type mismatch errors ( #684 )
...
The goal is to provide some context as to why the type were mismatched. This
change only works for that case, on structs. This is the same a encoding/json. A
more general solution would be great, but this would require a broader change in
the decoder, which I don't think is necessary at the moment.
Fixes #628
2021-11-24 20:43:56 -05:00
79e78b234c
Decoder: fix panic on table array behind a pointer ( #682 )
...
Fixes #677
2021-11-24 18:50:04 -05:00
64fe47161f
API: Encoder and Decoder options are chainable ( #670 )
...
Fixes #583
2021-11-13 19:04:53 -05:00
dc1740d473
Decode: code cleanup for struct cache ( #659 )
2021-11-07 18:35:30 -05:00
c871a61015
unmarshal: use UnmarshalText for any type ( #642 )
...
Not only structs can implement TextUnmarshaler.
Fixes #564
2021-10-28 17:02:47 -04:00
d0d001625c
unmarshal: don't panic when storing table in slice ( #641 )
...
New error message:
```
toml: cannot store a table in a slice
1| [things]
| ~~~~~~ cannot store a table in a slice
2| foo = "bar"
```
Fixes #623
2021-10-25 16:47:10 -04:00
64941b99e2
unmarshal: empty document results in map ( #640 )
...
Fixes #602
2021-10-25 15:55:54 -04:00
ed02a1f192
seen: check for explicit tables on dotted keys ( #639 )
...
The TOML spec is being clarified to say that dotted keys "define" their
intermediate tables. Therefore the seen tracker needs to verify that none of
them reference an explicit table.
Also added a missing seen expression check for key-values parsed as part of a
table section.
See https://github.com/toml-lang/toml/issues/846
2021-10-22 23:25:28 -04:00
cd54472d03
Validate UTF-8 ( #629 )
2021-10-15 19:13:21 -04:00
ee9b902222
unmarshal: convert ints if target type is compatible ( #594 )
...
This is required to support custom types.
Fixes #590
2021-09-09 21:25:14 -04:00
1230ca485e
unmarshal: make copy of non addressable values ( #576 )
...
When unmarshaling into a nested struct in a map, the value is not
addressable. In that case, make a copy of it and modify it instead.
Fixes #575
2021-08-31 20:22:38 -04:00
8be357dfa1
Add LocalTime to interface{} decode support ( #567 )
...
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes >
2021-07-21 17:50:12 +02:00
f6b38c33b7
Provide own implementation of Local* ( #558 )
...
* Reduces the public API.
* Reuses optimized parsing functions.
* Removes reliance on Google code under Apache license.
2021-06-08 20:27:05 -04:00
773f10110c
Unmarshal recursive structs ( #557 )
...
Co-authored-by: Nabetani <takenori@nabetani.sakura.ne.jp >
2021-06-08 14:22:39 -04:00
618f0181ac
AST Tweaks ( #551 )
...
* Use pointers instead of copying around ast.Node
Node is a 56B struct that is constantly in the hot path. Passing nodes
around by copy had a cost that started to add up. This change replaces
them by pointers. Using unsafe pointer arithmetic and converting
sibling/child indexes to relative offsets, it removes the need to carry
around a pointer to the root of the tree. This saves 8B per Node. This
space will be used to store an extra []byte slice to provide contextual
error handling on all nodes, including the ones whose data is different
than the raw input (for example: strings with escaped characters), while
staying under the size of a cache line.
* Remove conditional
* Add Raw to track range in data for parsed values
* Simplify reference tracking
2021-06-03 21:48:51 -04:00
f3bb20ea79
Benchmark marshal ( #550 )
2021-06-02 09:29:19 -04:00
250e073408
Stack-based unmarshaler ( #546 )
...
* Benchmark script
* Rewrite unmarshaler using the stack
Instead of tracking the build chain using `target`s, use the stack
instead.
Working and most benchmarks look good, but regression on structs unmarshalling.
~60% slower on ReferenceFile/struct.
* Shortcut to check if last node of iterator
* Remove unecessary pointer allocation
* Skip over unused keys without marking them as seen
* Add some tests
* Fix mktemp on macos
2021-05-31 12:14:13 -04:00
11f022ab09
Fix parser skipping over the whole file ( #547 )
...
* Add test for ReferenceFile/struct
* Stop skipping after table
* Add .gitattributes to force LF encoding
* Fix the reference file
2021-05-30 18:53:07 -04:00
95c701b253
Increase test coverage ( #538 )
...
Also fix array in map bug.
2021-05-10 20:17:05 -04:00
45ea20024b
Readme ( #535 )
2021-05-08 17:03:51 -04:00
ea225df3ed
v2: errors ( #534 )
...
```
name old time/op new time/op delta
UnmarshalDataset/config-32 86.7ms ± 2% 87.5ms ± 2% ~ (p=0.113 n=9+10)
UnmarshalDataset/canada-32 129ms ± 4% 106ms ± 3% -17.94% (p=0.000 n=10+10)
UnmarshalDataset/citm_catalog-32 59.4ms ± 5% 58.7ms ± 5% ~ (p=0.393 n=10+10)
UnmarshalDataset/twitter-32 27.0ms ± 7% 26.9ms ± 6% ~ (p=0.720 n=10+9)
UnmarshalDataset/code-32 326ms ± 4% 322ms ± 7% ~ (p=0.661 n=9+10)
UnmarshalDataset/example-32 510µs ±11% 526µs ± 7% ~ (p=0.182 n=10+9)
UnmarshalSimple-32 1.41µs ± 6% 1.41µs ± 4% ~ (p=0.736 n=10+9)
ReferenceFile-32 45.6µs ± 3% 43.9µs ±10% ~ (p=0.089 n=10+10)
name old speed new speed delta
UnmarshalDataset/config-32 12.1MB/s ± 2% 12.0MB/s ± 2% ~ (p=0.108 n=9+10)
UnmarshalDataset/canada-32 17.1MB/s ± 4% 20.9MB/s ± 3% +21.86% (p=0.000 n=10+10)
UnmarshalDataset/citm_catalog-32 9.41MB/s ± 5% 9.51MB/s ± 5% ~ (p=0.362 n=10+10)
UnmarshalDataset/twitter-32 16.4MB/s ± 8% 16.5MB/s ± 6% ~ (p=0.704 n=10+9)
UnmarshalDataset/code-32 8.24MB/s ± 4% 8.34MB/s ± 7% ~ (p=0.675 n=9+10)
UnmarshalDataset/example-32 15.9MB/s ±11% 15.4MB/s ± 7% ~ (p=0.182 n=10+9)
ReferenceFile-32 115MB/s ± 4% 120MB/s ±10% ~ (p=0.085 n=10+10)
name old alloc/op new alloc/op delta
UnmarshalDataset/config-32 16.9MB ± 0% 16.9MB ± 0% -0.02% (p=0.000 n=10+10)
UnmarshalDataset/canada-32 76.8MB ± 0% 74.3MB ± 0% -3.31% (p=0.000 n=10+10)
UnmarshalDataset/citm_catalog-32 37.3MB ± 0% 37.1MB ± 0% -0.60% (p=0.000 n=9+10)
UnmarshalDataset/twitter-32 15.6MB ± 0% 15.6MB ± 0% -0.09% (p=0.000 n=10+10)
UnmarshalDataset/code-32 60.2MB ± 0% 59.3MB ± 0% -1.51% (p=0.000 n=10+9)
UnmarshalDataset/example-32 238kB ± 0% 238kB ± 0% -0.18% (p=0.000 n=10+10)
ReferenceFile-32 11.8kB ± 0% 11.8kB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
UnmarshalDataset/config-32 653k ± 0% 645k ± 0% -1.20% (p=0.000 n=10+6)
UnmarshalDataset/canada-32 1.01M ± 0% 0.90M ± 0% -11.04% (p=0.000 n=9+10)
UnmarshalDataset/citm_catalog-32 384k ± 0% 370k ± 0% -3.75% (p=0.000 n=10+10)
UnmarshalDataset/twitter-32 160k ± 0% 157k ± 0% -1.32% (p=0.000 n=10+10)
UnmarshalDataset/code-32 2.97M ± 0% 2.91M ± 0% -2.15% (p=0.000 n=10+7)
UnmarshalDataset/example-32 3.69k ± 0% 3.63k ± 0% -1.52% (p=0.000 n=10+10)
ReferenceFile-32 253 ± 0% 253 ± 0% ~ (all equal)
```
2021-05-08 16:04:25 -04:00
3f2bb0b363
golangci-lint ( #530 )
2021-05-06 22:29:21 -04:00
201d5dd422
golangci-lint: misc ( #529 )
2021-04-27 20:29:00 -04:00
ee102a3528
decoder: fix time fractional parsing
2021-04-20 23:16:08 -04:00
9b67e40640
decoder: strict mode ( #512 )
2021-04-20 21:26:22 -04:00
466bfe8664
encoder: inline tables create map in nil interface ( #496 )
...
Co-authored-by: Thomas Pelletier <pelletier.thomas@gmail.com >
2021-04-09 09:02:00 -04:00
ca41df4a59
encoder: only create empty map when target exists
2021-04-08 19:40:34 -04:00
37714006b6
V2 Marshaler MVP ( #495 )
2021-04-08 10:07:29 -04:00
275e366c17
decoder: handle casting local date into time.Time
...
Refs #494
2021-04-08 08:00:31 -04:00
32da85ab11
Decoding error position tracking
2021-03-30 21:43:57 -04:00
18d45c446b
wip: decoder errors
2021-03-30 19:52:02 -04:00
2714786b37
Add decoder interface
2021-03-29 21:30:41 -04:00
da21b0aecf
wip: correctness pass on the AST
2021-03-28 22:12:19 -04:00
390927a0cd
Reuse AST storage between top-level expressions
...
```
Comparing:
old: v2-wip/1da2fc7 (2021-03-25 20:38:05 -0400 -0400)
run: v2-wip/3f23ab9 (2021-03-25 22:35:06 -0400 -0400)
-----------------------------------------------------------
name old time/op new time/op delta
UnmarshalSimple/v2-8 700ns ± 3% 705ns ± 2% ~ (p=0.690 n=5+5)
UnmarshalSimple/v1-8 3.85µs ± 1% 4.02µs ± 4% +4.19% (p=0.032 n=5+5)
UnmarshalSimple/bs-8 2.34µs ± 2% 2.38µs ± 3% ~ (p=0.310 n=5+5)
ReferenceFile/v2-8 32.2µs ±13% 23.9µs ± 1% -25.79% (p=0.008 n=5+5)
ReferenceFile/v1-8 270µs ± 2% 264µs ± 2% ~ (p=0.095 n=5+5)
ReferenceFile/bs-8 291µs ± 0% 294µs ± 0% +0.88% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
ReferenceFile/v2-8 37.1kB ± 0% 6.7kB ± 0% -81.91% (p=0.008 n=5+5)
ReferenceFile/v1-8 131kB ± 0% 131kB ± 0% ~ (p=0.444 n=5+5)
ReferenceFile/bs-8 80.8kB ± 0% 80.8kB ± 0% ~ (p=0.571 n=5+5)
name old allocs/op new allocs/op delta
ReferenceFile/v2-8 152 ± 0% 148 ± 0% -2.63% (p=0.008 n=5+5)
ReferenceFile/v1-8 2.65k ± 0% 2.65k ± 0% ~ (all equal)
ReferenceFile/bs-8 1.73k ± 0% 1.73k ± 0% ~ (all equal)
~/s/g/p/g/benchmark$ go test -bench=.
goos: linux
goarch: amd64
pkg: github.com/pelletier/go-toml/v2/benchmark
cpu: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
BenchmarkUnmarshalSimple/v2-8 1692444 710.7 ns/op
BenchmarkUnmarshalSimple/v1-8 307609 3862 ns/op
BenchmarkUnmarshalSimple/bs-8 520429 2285 ns/op
BenchmarkReferenceFile/v2-8 50395 24006 ns/op 6704 B/op 148 allocs/op
BenchmarkReferenceFile/v1-8 4144 264655 ns/op 130567 B/op 2649 allocs/op
BenchmarkReferenceFile/bs-8 3969 293635 ns/op 80784 B/op 1729 allocs/op
PASS
ok github.com/pelletier/go-toml/v2/benchmark 8.143s
```
2021-03-25 22:37:16 -04:00
1bae751a45
Linear array storage for AST
2021-03-25 19:56:02 -04:00
43fc2fa552
Factor pointer handling
2021-03-24 21:05:44 -04:00
dd5837651d
Support TextUnmarshaler
2021-03-24 21:02:02 -04:00
Thomas Pelletier