19751e8a51
Missing performance section
2021-10-28 19:10:43 -04:00
925f214125
Add GitHub release configuration ( #644 )
2021-10-28 19:06:14 -04:00
39f893ad99
Multiline strings fixes ( #643 )
...
* scanner: allow multiline strings to end with "" or ''
* parser: trim all whitespaces after \ in multiline
2021-10-28 18:26:34 -04:00
c871a61015
unmarshal: use UnmarshalText for any type ( #642 )
...
Not only structs can implement TextUnmarshaler.
Fixes #564
2021-10-28 17:02:47 -04:00
d0d001625c
unmarshal: don't panic when storing table in slice ( #641 )
...
New error message:
```
toml: cannot store a table in a slice
1| [things]
| ~~~~~~ cannot store a table in a slice
2| foo = "bar"
```
Fixes #623
2021-10-25 16:47:10 -04:00
64941b99e2
unmarshal: empty document results in map ( #640 )
...
Fixes #602
2021-10-25 15:55:54 -04:00
ed02a1f192
seen: check for explicit tables on dotted keys ( #639 )
...
The TOML spec is being clarified to say that dotted keys "define" their
intermediate tables. Therefore the seen tracker needs to verify that none of
them reference an explicit table.
Also added a missing seen expression check for key-values parsed as part of a
table section.
See https://github.com/toml-lang/toml/issues/846
2021-10-22 23:25:28 -04:00
4d7c9ddac7
Floats and integers parsing fixes ( #638 )
...
* parser: fix scan of float with exp but no decimal
* decoder: validate leading zeros for decimals
2021-10-22 22:25:56 -04:00
feb1830dcc
tomltest: enable TestTOMLTest_Valid_Comment_Tricky
2021-10-21 22:30:58 -04:00
1c33d6ce20
tomltest: custom comparison functions ( #637 )
2021-10-21 22:29:04 -04:00
3000471a12
parser: improve floats validation ( #636 )
2021-10-20 08:49:28 -04:00
1f33a6a476
tomltest: enable Valid_Datetime_LocalTime
2021-10-19 21:20:45 -04:00
2700aad5d2
tomltest: run UTF8 tests ( #634 )
2021-10-19 16:00:56 -04:00
7ccaa2744e
tomltest: unmarshal JSONs for tests ( #633 )
...
Comparing the output and the expected results byte-wise means we get false
negative when order doesn't matter (for example the ValidTableKeyword test).
2021-10-19 15:29:49 -04:00
df4bb061f8
time: follow RFC3339 spec for datetime ( #632 )
2021-10-18 09:56:07 -04:00
9e81ce1c33
Create SECURITY.md
2021-10-17 20:57:34 -04:00
a23850f29b
decode: preserve nanosecond precision when decoding time ( #626 )
...
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes >
2021-10-17 20:43:29 -04:00
76f53c857b
unmarshal: validate date ( #622 )
2021-10-17 20:18:20 -04:00
85f5d567e4
parser: validate invalid ASCII control characters
2021-10-16 07:41:12 -04:00
bd5cba0b0b
Update benchmarks readme ( #630 )
...
* Fix ci.sh for new benchmarks
Nice + taskset are more stable on my machine. We want to excude non high-level
benchmarks. BurntSushi/toml now supports canada.toml.
* Update latest benchmarks in README
2021-10-15 19:53:40 -04:00
cd54472d03
Validate UTF-8 ( #629 )
2021-10-15 19:13:21 -04:00
cc0d1a90ff
testgen: skip currently failing tests ( #627 )
2021-10-14 11:14:44 -04:00
4984dcb5e9
encode: ensure floats have decimal point ( #615 )
...
Fixes #571
Co-authored-by: Sterling Hanenkamp <sterling@ziprecruiter.com >
2021-10-14 08:34:54 -04:00
86632bc190
parser: fail when missing array separator ( #616 )
...
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes >
2021-10-14 08:26:29 -04:00
d25eec183f
gotoml-test-decoder: add toml-test decoder command ( #619 )
2021-10-14 08:14:34 -04:00
e96746311c
decoder: fix panic date time should have a timezone ( #614 )
...
Fixes #596
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes >
2021-10-06 21:24:25 -04:00
62acca2b68
tomltestgen: add toml-test unit test generation command ( #610 )
...
Tests are hidden behind a "testsuite" build tag for now since many tests
are failing. Use `go test -tags testsuite` to activate.
Use `go generate` to regenerate toml_testgen_test.go.
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes >
2021-10-03 22:15:30 -04:00
476492a85c
unmarshal: support lowercase 'T' and 'Z' in date-time parsing ( #601 )
...
RFC3399 allows for lowercase 't' and 'z' in date-time values.
Fixes #600
2021-09-25 10:02:23 -07:00
ee9b902222
unmarshal: convert ints if target type is compatible ( #594 )
...
This is required to support custom types.
Fixes #590
2021-09-09 21:25:14 -04:00
fa56f48daf
parser: don't overflow when parsing bad times ( #593 )
...
Fixes #585
2021-09-09 11:59:37 -04:00
f34c9c332f
scanner: fix error reporting for last comments ( #591 )
...
When an invalid TOML expression ends with a comment before the end of
file, the decode error would take a nil from scanComment, which is not
part of the document.
Fixes #588
2021-09-08 21:54:30 -04:00
a0d685d482
unmarshal: don't crash on unterminated inline table ( #587 )
...
Fixes #586
2021-09-07 20:08:59 -04:00
4a5ae9e81e
errors: fix context generation with only one line
2021-09-07 10:36:22 -04:00
7e2fa1bc80
unmarshal: fix non-terminated array error
...
Fixes #581
2021-09-07 10:36:22 -04:00
40cfb6f458
parser: don't crash on unterminated table key ( #580 )
...
* parser: don't crash on unterminated table key
Fixes #579
* parser: fix format of error returned by expect
EOF was missing the format string and %U is not very human friendly.
2021-09-06 12:18:45 -04:00
1230ca485e
unmarshal: make copy of non addressable values ( #576 )
...
When unmarshaling into a nested struct in a map, the value is not
addressable. In that case, make a copy of it and modify it instead.
Fixes #575
2021-08-31 20:22:38 -04:00
69ab7e10d1
Go 1.17 release ( #574 )
...
Minimum supported version: Go 1.16.
2021-08-17 09:43:52 -04:00
fa07960695
Add installation instructions ( #572 )
2021-07-27 18:12:44 -04:00
8be357dfa1
Add LocalTime to interface{} decode support ( #567 )
...
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes >
2021-07-21 17:50:12 +02:00
a93b34d984
Unicode parsing optimization ( #568 )
...
Inline call to hexToRune and uses specialized parsing, as found in encoding/json.
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes >
2021-07-21 10:50:03 +02:00
9c24fbeaad
Set up Dependabot for GitHub actions and docker ( #570 )
2021-07-20 16:54:26 +02:00
f6b38c33b7
Provide own implementation of Local* ( #558 )
...
* Reduces the public API.
* Reuses optimized parsing functions.
* Removes reliance on Google code under Apache license.
2021-06-08 20:27:05 -04:00
773f10110c
Unmarshal recursive structs ( #557 )
...
Co-authored-by: Nabetani <takenori@nabetani.sakura.ne.jp >
2021-06-08 14:22:39 -04:00
618f0181ac
AST Tweaks ( #551 )
...
* Use pointers instead of copying around ast.Node
Node is a 56B struct that is constantly in the hot path. Passing nodes
around by copy had a cost that started to add up. This change replaces
them by pointers. Using unsafe pointer arithmetic and converting
sibling/child indexes to relative offsets, it removes the need to carry
around a pointer to the root of the tree. This saves 8B per Node. This
space will be used to store an extra []byte slice to provide contextual
error handling on all nodes, including the ones whose data is different
than the raw input (for example: strings with escaped characters), while
staying under the size of a cache line.
* Remove conditional
* Add Raw to track range in data for parsed values
* Simplify reference tracking
2021-06-03 21:48:51 -04:00
f3bb20ea79
Benchmark marshal ( #550 )
2021-06-02 09:29:19 -04:00
b0d6c62255
Don't use bytes.Buffer when not necessary ( #549 )
...
When parsing strings, they can be referenced directly from the document
when they don't contain escaped characters. This avoids paying to cost
of allocating (and sometimes growing) the bytes buffer unecessarily.
2021-06-01 09:51:59 -04:00
b202375414
Add benchmarks results to readme ( #548 )
2021-06-01 09:10:17 -04:00
250e073408
Stack-based unmarshaler ( #546 )
...
* Benchmark script
* Rewrite unmarshaler using the stack
Instead of tracking the build chain using `target`s, use the stack
instead.
Working and most benchmarks look good, but regression on structs unmarshalling.
~60% slower on ReferenceFile/struct.
* Shortcut to check if last node of iterator
* Remove unecessary pointer allocation
* Skip over unused keys without marking them as seen
* Add some tests
* Fix mktemp on macos
2021-05-31 12:14:13 -04:00
11f022ab09
Fix parser skipping over the whole file ( #547 )
...
* Add test for ReferenceFile/struct
* Stop skipping after table
* Add .gitattributes to force LF encoding
* Fix the reference file
2021-05-30 18:53:07 -04:00
840df4a229
seen tracker: use array for storage ( #545 )
...
name old time/op new time/op delta
UnmarshalDataset/config-32 81.2ms ± 3% 77.8ms ± 3% -4.25% (p=0.000 n=20+19)
UnmarshalDataset/canada-32 104ms ± 5% 105ms ± 4% ~ (p=0.102 n=20+20)
UnmarshalDataset/citm_catalog-32 57.5ms ± 5% 59.0ms ± 5% +2.54% (p=0.033 n=20+20)
UnmarshalDataset/twitter-32 25.7ms ± 7% 28.1ms ± 5% +9.33% (p=0.000 n=20+20)
UnmarshalDataset/code-32 305ms ± 6% 292ms ± 5% -4.29% (p=0.000 n=20+19)
UnmarshalDataset/example-32 519µs ± 6% 522µs ± 5% ~ (p=0.659 n=20+20)
UnmarshalSimple/struct-32 1.44µs ± 1% 1.17µs ± 6% -18.78% (p=0.000 n=14+20)
UnmarshalSimple/map-32 2.30µs ± 4% 1.99µs ± 4% -13.65% (p=0.000 n=20+19)
ReferenceFile/struct-32 44.1µs ± 4% 38.1µs ± 5% -13.61% (p=0.000 n=18+20)
ReferenceFile/map-32 197µs ± 7% 189µs ± 5% -3.91% (p=0.000 n=20+20)
HugoFrontMatter-32 45.9µs ± 6% 39.3µs ± 6% -14.46% (p=0.000 n=19+20)
name old speed new speed delta
UnmarshalDataset/config-32 12.9MB/s ± 3% 13.5MB/s ± 3% +4.42% (p=0.000 n=20+19)
UnmarshalDataset/canada-32 21.1MB/s ± 5% 20.9MB/s ± 4% ~ (p=0.101 n=20+20)
UnmarshalDataset/citm_catalog-32 9.72MB/s ± 6% 9.47MB/s ± 5% -2.53% (p=0.031 n=20+20)
UnmarshalDataset/twitter-32 17.2MB/s ± 7% 15.8MB/s ± 5% -8.57% (p=0.000 n=20+20)
UnmarshalDataset/code-32 8.81MB/s ± 7% 9.20MB/s ± 5% +4.47% (p=0.000 n=20+19)
UnmarshalDataset/example-32 15.6MB/s ± 6% 15.5MB/s ± 5% ~ (p=0.644 n=20+20)
UnmarshalSimple/struct-32 7.61MB/s ± 1% 9.39MB/s ± 7% +23.33% (p=0.000 n=15+20)
UnmarshalSimple/map-32 4.78MB/s ± 4% 5.54MB/s ± 5% +15.85% (p=0.000 n=20+19)
ReferenceFile/struct-32 119MB/s ± 4% 138MB/s ± 5% +15.79% (p=0.000 n=18+20)
ReferenceFile/map-32 26.6MB/s ± 7% 27.7MB/s ± 5% +4.06% (p=0.000 n=20+20)
HugoFrontMatter-32 11.9MB/s ± 6% 13.9MB/s ± 6% +16.91% (p=0.000 n=19+20)
name old alloc/op new alloc/op delta
UnmarshalDataset/config-32 16.9MB ± 0% 13.9MB ± 0% -17.48% (p=0.000 n=18+18)
UnmarshalDataset/canada-32 74.3MB ± 0% 74.3MB ± 0% -0.00% (p=0.000 n=20+20)
UnmarshalDataset/citm_catalog-32 37.3MB ± 0% 37.3MB ± 0% +0.11% (p=0.000 n=20+20)
UnmarshalDataset/twitter-32 15.6MB ± 0% 15.6MB ± 0% ~ (p=0.211 n=19+20)
UnmarshalDataset/code-32 59.5MB ± 0% 52.4MB ± 0% -11.96% (p=0.000 n=19+20)
UnmarshalDataset/example-32 238kB ± 0% 239kB ± 0% +0.02% (p=0.000 n=18+20)
UnmarshalSimple/struct-32 981B ± 0% 709B ± 0% -27.73% (p=0.000 n=20+20)
UnmarshalSimple/map-32 1.45kB ± 0% 1.17kB ± 0% -18.82% (p=0.000 n=20+20)
ReferenceFile/struct-32 11.8kB ± 0% 9.7kB ± 0% -17.64% (p=0.000 n=20+20)
ReferenceFile/map-32 51.5kB ± 0% 52.2kB ± 0% +1.30% (p=0.000 n=20+17)
HugoFrontMatter-32 12.1kB ± 0% 11.1kB ± 0% -7.97% (p=0.000 n=20+19)
name old allocs/op new allocs/op delta
UnmarshalDataset/config-32 645k ± 0% 557k ± 0% -13.76% (p=0.000 n=20+16)
UnmarshalDataset/canada-32 896k ± 0% 896k ± 0% -0.00% (p=0.000 n=20+20)
UnmarshalDataset/citm_catalog-32 380k ± 0% 377k ± 0% -0.75% (p=0.000 n=19+20)
UnmarshalDataset/twitter-32 158k ± 0% 158k ± 0% -0.01% (p=0.000 n=18+18)
UnmarshalDataset/code-32 2.92M ± 0% 2.57M ± 0% -11.87% (p=0.000 n=20+20)
UnmarshalDataset/example-32 3.66k ± 0% 3.64k ± 0% -0.63% (p=0.000 n=20+20)
UnmarshalSimple/struct-32 13.0 ± 0% 10.0 ± 0% -23.08% (p=0.000 n=20+20)
UnmarshalSimple/map-32 22.0 ± 0% 19.0 ± 0% -13.64% (p=0.000 n=20+20)
ReferenceFile/struct-32 253 ± 0% 155 ± 0% -38.74% (p=0.000 n=20+20)
ReferenceFile/map-32 1.67k ± 0% 1.44k ± 0% -14.23% (p=0.000 n=20+20)
HugoFrontMatter-32 357 ± 0% 313 ± 0% -12.32% (p=0.000 n=20+20)
2021-05-26 18:47:00 -04:00
Thomas Pelletier