250e073408
Stack-based unmarshaler ( #546 )
...
* Benchmark script
* Rewrite unmarshaler using the stack
Instead of tracking the build chain using `target`s, use the stack
instead.
Working and most benchmarks look good, but regression on structs unmarshalling.
~60% slower on ReferenceFile/struct.
* Shortcut to check if last node of iterator
* Remove unecessary pointer allocation
* Skip over unused keys without marking them as seen
* Add some tests
* Fix mktemp on macos
2021-05-31 12:14:13 -04:00
840df4a229
seen tracker: use array for storage ( #545 )
...
name old time/op new time/op delta
UnmarshalDataset/config-32 81.2ms ± 3% 77.8ms ± 3% -4.25% (p=0.000 n=20+19)
UnmarshalDataset/canada-32 104ms ± 5% 105ms ± 4% ~ (p=0.102 n=20+20)
UnmarshalDataset/citm_catalog-32 57.5ms ± 5% 59.0ms ± 5% +2.54% (p=0.033 n=20+20)
UnmarshalDataset/twitter-32 25.7ms ± 7% 28.1ms ± 5% +9.33% (p=0.000 n=20+20)
UnmarshalDataset/code-32 305ms ± 6% 292ms ± 5% -4.29% (p=0.000 n=20+19)
UnmarshalDataset/example-32 519µs ± 6% 522µs ± 5% ~ (p=0.659 n=20+20)
UnmarshalSimple/struct-32 1.44µs ± 1% 1.17µs ± 6% -18.78% (p=0.000 n=14+20)
UnmarshalSimple/map-32 2.30µs ± 4% 1.99µs ± 4% -13.65% (p=0.000 n=20+19)
ReferenceFile/struct-32 44.1µs ± 4% 38.1µs ± 5% -13.61% (p=0.000 n=18+20)
ReferenceFile/map-32 197µs ± 7% 189µs ± 5% -3.91% (p=0.000 n=20+20)
HugoFrontMatter-32 45.9µs ± 6% 39.3µs ± 6% -14.46% (p=0.000 n=19+20)
name old speed new speed delta
UnmarshalDataset/config-32 12.9MB/s ± 3% 13.5MB/s ± 3% +4.42% (p=0.000 n=20+19)
UnmarshalDataset/canada-32 21.1MB/s ± 5% 20.9MB/s ± 4% ~ (p=0.101 n=20+20)
UnmarshalDataset/citm_catalog-32 9.72MB/s ± 6% 9.47MB/s ± 5% -2.53% (p=0.031 n=20+20)
UnmarshalDataset/twitter-32 17.2MB/s ± 7% 15.8MB/s ± 5% -8.57% (p=0.000 n=20+20)
UnmarshalDataset/code-32 8.81MB/s ± 7% 9.20MB/s ± 5% +4.47% (p=0.000 n=20+19)
UnmarshalDataset/example-32 15.6MB/s ± 6% 15.5MB/s ± 5% ~ (p=0.644 n=20+20)
UnmarshalSimple/struct-32 7.61MB/s ± 1% 9.39MB/s ± 7% +23.33% (p=0.000 n=15+20)
UnmarshalSimple/map-32 4.78MB/s ± 4% 5.54MB/s ± 5% +15.85% (p=0.000 n=20+19)
ReferenceFile/struct-32 119MB/s ± 4% 138MB/s ± 5% +15.79% (p=0.000 n=18+20)
ReferenceFile/map-32 26.6MB/s ± 7% 27.7MB/s ± 5% +4.06% (p=0.000 n=20+20)
HugoFrontMatter-32 11.9MB/s ± 6% 13.9MB/s ± 6% +16.91% (p=0.000 n=19+20)
name old alloc/op new alloc/op delta
UnmarshalDataset/config-32 16.9MB ± 0% 13.9MB ± 0% -17.48% (p=0.000 n=18+18)
UnmarshalDataset/canada-32 74.3MB ± 0% 74.3MB ± 0% -0.00% (p=0.000 n=20+20)
UnmarshalDataset/citm_catalog-32 37.3MB ± 0% 37.3MB ± 0% +0.11% (p=0.000 n=20+20)
UnmarshalDataset/twitter-32 15.6MB ± 0% 15.6MB ± 0% ~ (p=0.211 n=19+20)
UnmarshalDataset/code-32 59.5MB ± 0% 52.4MB ± 0% -11.96% (p=0.000 n=19+20)
UnmarshalDataset/example-32 238kB ± 0% 239kB ± 0% +0.02% (p=0.000 n=18+20)
UnmarshalSimple/struct-32 981B ± 0% 709B ± 0% -27.73% (p=0.000 n=20+20)
UnmarshalSimple/map-32 1.45kB ± 0% 1.17kB ± 0% -18.82% (p=0.000 n=20+20)
ReferenceFile/struct-32 11.8kB ± 0% 9.7kB ± 0% -17.64% (p=0.000 n=20+20)
ReferenceFile/map-32 51.5kB ± 0% 52.2kB ± 0% +1.30% (p=0.000 n=20+17)
HugoFrontMatter-32 12.1kB ± 0% 11.1kB ± 0% -7.97% (p=0.000 n=20+19)
name old allocs/op new allocs/op delta
UnmarshalDataset/config-32 645k ± 0% 557k ± 0% -13.76% (p=0.000 n=20+16)
UnmarshalDataset/canada-32 896k ± 0% 896k ± 0% -0.00% (p=0.000 n=20+20)
UnmarshalDataset/citm_catalog-32 380k ± 0% 377k ± 0% -0.75% (p=0.000 n=19+20)
UnmarshalDataset/twitter-32 158k ± 0% 158k ± 0% -0.01% (p=0.000 n=18+18)
UnmarshalDataset/code-32 2.92M ± 0% 2.57M ± 0% -11.87% (p=0.000 n=20+20)
UnmarshalDataset/example-32 3.66k ± 0% 3.64k ± 0% -0.63% (p=0.000 n=20+20)
UnmarshalSimple/struct-32 13.0 ± 0% 10.0 ± 0% -23.08% (p=0.000 n=20+20)
UnmarshalSimple/map-32 22.0 ± 0% 19.0 ± 0% -13.64% (p=0.000 n=20+20)
ReferenceFile/struct-32 253 ± 0% 155 ± 0% -38.74% (p=0.000 n=20+20)
ReferenceFile/map-32 1.67k ± 0% 1.44k ± 0% -14.23% (p=0.000 n=20+20)
HugoFrontMatter-32 357 ± 0% 313 ± 0% -12.32% (p=0.000 n=20+20)
2021-05-26 18:47:00 -04:00
ea225df3ed
v2: errors ( #534 )
...
```
name old time/op new time/op delta
UnmarshalDataset/config-32 86.7ms ± 2% 87.5ms ± 2% ~ (p=0.113 n=9+10)
UnmarshalDataset/canada-32 129ms ± 4% 106ms ± 3% -17.94% (p=0.000 n=10+10)
UnmarshalDataset/citm_catalog-32 59.4ms ± 5% 58.7ms ± 5% ~ (p=0.393 n=10+10)
UnmarshalDataset/twitter-32 27.0ms ± 7% 26.9ms ± 6% ~ (p=0.720 n=10+9)
UnmarshalDataset/code-32 326ms ± 4% 322ms ± 7% ~ (p=0.661 n=9+10)
UnmarshalDataset/example-32 510µs ±11% 526µs ± 7% ~ (p=0.182 n=10+9)
UnmarshalSimple-32 1.41µs ± 6% 1.41µs ± 4% ~ (p=0.736 n=10+9)
ReferenceFile-32 45.6µs ± 3% 43.9µs ±10% ~ (p=0.089 n=10+10)
name old speed new speed delta
UnmarshalDataset/config-32 12.1MB/s ± 2% 12.0MB/s ± 2% ~ (p=0.108 n=9+10)
UnmarshalDataset/canada-32 17.1MB/s ± 4% 20.9MB/s ± 3% +21.86% (p=0.000 n=10+10)
UnmarshalDataset/citm_catalog-32 9.41MB/s ± 5% 9.51MB/s ± 5% ~ (p=0.362 n=10+10)
UnmarshalDataset/twitter-32 16.4MB/s ± 8% 16.5MB/s ± 6% ~ (p=0.704 n=10+9)
UnmarshalDataset/code-32 8.24MB/s ± 4% 8.34MB/s ± 7% ~ (p=0.675 n=9+10)
UnmarshalDataset/example-32 15.9MB/s ±11% 15.4MB/s ± 7% ~ (p=0.182 n=10+9)
ReferenceFile-32 115MB/s ± 4% 120MB/s ±10% ~ (p=0.085 n=10+10)
name old alloc/op new alloc/op delta
UnmarshalDataset/config-32 16.9MB ± 0% 16.9MB ± 0% -0.02% (p=0.000 n=10+10)
UnmarshalDataset/canada-32 76.8MB ± 0% 74.3MB ± 0% -3.31% (p=0.000 n=10+10)
UnmarshalDataset/citm_catalog-32 37.3MB ± 0% 37.1MB ± 0% -0.60% (p=0.000 n=9+10)
UnmarshalDataset/twitter-32 15.6MB ± 0% 15.6MB ± 0% -0.09% (p=0.000 n=10+10)
UnmarshalDataset/code-32 60.2MB ± 0% 59.3MB ± 0% -1.51% (p=0.000 n=10+9)
UnmarshalDataset/example-32 238kB ± 0% 238kB ± 0% -0.18% (p=0.000 n=10+10)
ReferenceFile-32 11.8kB ± 0% 11.8kB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
UnmarshalDataset/config-32 653k ± 0% 645k ± 0% -1.20% (p=0.000 n=10+6)
UnmarshalDataset/canada-32 1.01M ± 0% 0.90M ± 0% -11.04% (p=0.000 n=9+10)
UnmarshalDataset/citm_catalog-32 384k ± 0% 370k ± 0% -3.75% (p=0.000 n=10+10)
UnmarshalDataset/twitter-32 160k ± 0% 157k ± 0% -1.32% (p=0.000 n=10+10)
UnmarshalDataset/code-32 2.97M ± 0% 2.91M ± 0% -2.15% (p=0.000 n=10+7)
UnmarshalDataset/example-32 3.69k ± 0% 3.63k ± 0% -1.52% (p=0.000 n=10+10)
ReferenceFile-32 253 ± 0% 253 ± 0% ~ (all equal)
```
2021-05-08 16:04:25 -04:00
e443b4fdb8
encoder: support TextMarshaler ( #522 )
...
Fixes #521
2021-04-22 10:13:41 -04:00
ee102a3528
decoder: fix time fractional parsing
2021-04-20 23:16:08 -04:00
9b67e40640
decoder: strict mode ( #512 )
2021-04-20 21:26:22 -04:00
37714006b6
V2 Marshaler MVP ( #495 )
2021-04-08 10:07:29 -04:00
af00765ca0
Address golangci-lint warnings in unmarshal_imported_test.go ( #493 )
...
* refactor(tracker): Remove unreachable return
* refactor(unmarshal_imported_test): Mark unused camelCase test
golangci-lint indicates `TestUnmarshalCamelCaseKey` as unused (it's
currently skipped).
* refactor(unmarshal_imported_test): Mark unused type
golangci-lint indicates `customPointerMarshaler` as unused.
* refactor(unmarshal_imported_test): Mark unused type
golangci-lint indicates `textPointerMarshaler` as unused.
* refactor(unmarshal_imported_test): Mark unused type
golangci-lint indicates `precedentMarshaler` and its methods as
unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `testDurationToml2` as unused.
* refactor(unmarshal_imported_test): Mark unused type
golangci-lint indicates `testBadDuration` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `testDurationToml` as unused.
* refactor(unmarshal_imported_test): Mark unused type
golangci-lint indicates `testDuration` as unused.
* refactor(unmarshal_imported_test): Mark unused type
golangci-lint indicates `testDocCustomTag` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `testDocCustomTagData` as unused.
* refactor(unmarshal_imported_test): Mark unused type
golangci-lint indicates `testDocBasicsCustomTag` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `testDocBasicToml` as unused.
* refactor(unmarshal_imported_test): Mark unused type
golangci-lint indicates `structArrayNoTag` as unused.
* refactor(unmarshal_imported_test): Mark unused type
golangci-lint incorrectly indicates `check` as unused.
* refactor(unmarshal_imported_test): Mark unused struct field
golangci-lint indicates `testDoc.err` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `customMultilineTagTestToml` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `customCommentedTagTestToml` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `customCommentTagTestToml` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `customTagTestToml` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `mapsTestToml` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `mapsTestData` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `commentTestToml` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `nestedCustomMarshalerToml` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `nestedCustomMarshalerData` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `customMarshalerToml` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `mapTestDoc` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `quotedKeyMarshalTestToml` as unused.
* refactor(unmarshal_imported_test): Mark unused var
golangci-lint indicates `quotedKeyMarshalTestData` as unused.
2021-04-05 14:30:17 -04:00
32da85ab11
Decoding error position tracking
2021-03-30 21:43:57 -04:00
cf288a51c5
Wip errors reporting
2021-03-30 10:59:35 -04:00
72a1afdcb2
Add some unsafe helper to track errors
2021-03-29 22:33:28 -04:00
2ddbf6be6d
Implement duplicate and key types check
2021-03-29 10:45:50 -04:00
da21b0aecf
wip: correctness pass on the AST
2021-03-28 22:12:19 -04:00
390927a0cd
Reuse AST storage between top-level expressions
...
```
Comparing:
old: v2-wip/1da2fc7 (2021-03-25 20:38:05 -0400 -0400)
run: v2-wip/3f23ab9 (2021-03-25 22:35:06 -0400 -0400)
-----------------------------------------------------------
name old time/op new time/op delta
UnmarshalSimple/v2-8 700ns ± 3% 705ns ± 2% ~ (p=0.690 n=5+5)
UnmarshalSimple/v1-8 3.85µs ± 1% 4.02µs ± 4% +4.19% (p=0.032 n=5+5)
UnmarshalSimple/bs-8 2.34µs ± 2% 2.38µs ± 3% ~ (p=0.310 n=5+5)
ReferenceFile/v2-8 32.2µs ±13% 23.9µs ± 1% -25.79% (p=0.008 n=5+5)
ReferenceFile/v1-8 270µs ± 2% 264µs ± 2% ~ (p=0.095 n=5+5)
ReferenceFile/bs-8 291µs ± 0% 294µs ± 0% +0.88% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
ReferenceFile/v2-8 37.1kB ± 0% 6.7kB ± 0% -81.91% (p=0.008 n=5+5)
ReferenceFile/v1-8 131kB ± 0% 131kB ± 0% ~ (p=0.444 n=5+5)
ReferenceFile/bs-8 80.8kB ± 0% 80.8kB ± 0% ~ (p=0.571 n=5+5)
name old allocs/op new allocs/op delta
ReferenceFile/v2-8 152 ± 0% 148 ± 0% -2.63% (p=0.008 n=5+5)
ReferenceFile/v1-8 2.65k ± 0% 2.65k ± 0% ~ (all equal)
ReferenceFile/bs-8 1.73k ± 0% 1.73k ± 0% ~ (all equal)
~/s/g/p/g/benchmark$ go test -bench=.
goos: linux
goarch: amd64
pkg: github.com/pelletier/go-toml/v2/benchmark
cpu: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
BenchmarkUnmarshalSimple/v2-8 1692444 710.7 ns/op
BenchmarkUnmarshalSimple/v1-8 307609 3862 ns/op
BenchmarkUnmarshalSimple/bs-8 520429 2285 ns/op
BenchmarkReferenceFile/v2-8 50395 24006 ns/op 6704 B/op 148 allocs/op
BenchmarkReferenceFile/v1-8 4144 264655 ns/op 130567 B/op 2649 allocs/op
BenchmarkReferenceFile/bs-8 3969 293635 ns/op 80784 B/op 1729 allocs/op
PASS
ok github.com/pelletier/go-toml/v2/benchmark 8.143s
```
2021-03-25 22:37:16 -04:00
1d332cd112
Add documentation for the AST
2021-03-25 20:46:31 -04:00
9d3a912da0
Remove unused interface
...
Comparing:
old: v2-wip/17299c9 (2021-03-25 20:19:40 -0400 -0400)
run: v2-wip/1da2fc7 (2021-03-25 20:38:05 -0400 -0400)
-----------------------------------------------------------
name old time/op new time/op delta
UnmarshalSimple/v2-8 755ns ± 3% 700ns ± 3% -7.26% (p=0.008 n=5+5)
UnmarshalSimple/v1-8 3.87µs ± 0% 3.85µs ± 1% ~ (p=0.254 n=4+5)
UnmarshalSimple/bs-8 2.44µs ± 4% 2.34µs ± 2% ~ (p=0.056 n=5+5)
ReferenceFile/v2-8 33.5µs ± 7% 32.2µs ±13% ~ (p=0.421 n=5+5)
ReferenceFile/v1-8 269µs ± 3% 270µs ± 2% ~ (p=1.000 n=5+5)
ReferenceFile/bs-8 296µs ± 2% 291µs ± 0% ~ (p=0.095 n=5+5)
name old alloc/op new alloc/op delta
ReferenceFile/v2-8 38.9kB ± 0% 37.1kB ± 0% -4.77% (p=0.008 n=5+5)
ReferenceFile/v1-8 131kB ± 0% 131kB ± 0% ~ (all equal)
ReferenceFile/bs-8 80.8kB ± 0% 80.8kB ± 0% ~ (p=0.841 n=5+5)
name old allocs/op new allocs/op delta
ReferenceFile/v2-8 181 ± 0% 152 ± 0% -16.02% (p=0.008 n=5+5)
ReferenceFile/v1-8 2.65k ± 0% 2.65k ± 0% ~ (all equal)
ReferenceFile/bs-8 1.73k ± 0% 1.73k ± 0% ~ (all equal)
2021-03-25 20:38:45 -04:00
1bae751a45
Linear array storage for AST
2021-03-25 19:56:02 -04:00
ad538d97c9
Delete reflectbuild
2021-03-24 21:06:38 -04:00
dd5837651d
Support TextUnmarshaler
2021-03-24 21:02:02 -04:00
a0d031abec
Arrays support
2021-03-24 20:21:55 -04:00
a25f636a07
Add array support todo
2021-03-23 21:18:19 -04:00
bfeb32c9ce
Make unmarshal to interface{} consistent with encoding/json
2021-03-23 20:03:45 -04:00
c6f117c45d
Handle pointers in slices
2021-03-23 09:15:48 -04:00
e78ccff9a4
Fix parsing integer 0
2021-03-23 09:02:48 -04:00
fcc91f2618
Progress on date/times
2021-03-22 09:59:15 -04:00
8b34e54764
Improve DOT representation for AST
2021-03-18 22:10:31 -04:00
9ec4e86883
Handle struct field name variations
2021-03-18 20:42:41 -04:00
93a7b0d77d
Skip AST branches that don't exist in the target
2021-03-18 20:30:51 -04:00
3e8b8db786
Unmarshal into pointers
2021-03-18 20:02:32 -04:00
8957a768ef
Next stop: pointers
2021-03-18 19:52:11 -04:00
f9f9ccb777
Basic array table implementation
2021-03-16 10:24:19 -04:00
00b2f776a9
Replace branch with AST version
2021-03-15 08:46:35 -04:00
b8df31de84
Comment out date/time tests for now
2021-03-14 18:13:57 -04:00
590d674153
Unmarshal ints and floats
2021-03-14 18:06:34 -04:00
9a1cfcdd8e
Replace parser's int or float code with scanner
2021-03-14 17:22:53 -04:00
590d7faf65
Parser emits AST node for all kinds of strings
2021-03-14 16:16:29 -04:00
de035f0fed
Standard tables in parser
2021-03-14 16:11:23 -04:00
04925e4882
Handle bools
2021-03-14 15:52:22 -04:00
3760527218
Unmarshal tests
2021-03-13 23:42:38 -05:00
fa7ee6461a
Inline tables
2021-03-13 23:06:16 -05:00
fbf01f7683
Handle Table
2021-03-13 22:48:31 -05:00
a0548e793c
Unmarshal slices of strings
2021-03-13 22:07:36 -05:00
d8be04d4a8
Handle simple string slice
2021-03-13 18:45:03 -05:00
21d3e85fcc
Playing with an AST
...
Idea would be to build a light AST as a first pass, then have the
unmarshaler and Document parser do what they need with it.
2021-03-13 11:38:09 -05:00
93a74fca35
todo: inline tables
2021-03-08 21:59:43 -05:00
a1c9b661b4
Allocate slice if needed
2021-03-08 21:41:03 -05:00
87b9d1cf98
Handle overflows
2021-03-08 21:01:53 -05:00
90f3b658c6
Support type aliases
2021-03-08 20:27:04 -05:00
c35bcc5519
Convert returns pointer if a pointer is passed
2021-03-08 10:01:56 -05:00
f698c102c7
Convert should only handle specific types
2021-03-08 09:42:19 -05:00
Thomas Pelletier