ostiwe/go-toml
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
854 Commits 18 Branches 58 Tags
cursor/unsafe-package-removal-5147
Commit Graph

854 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Thomas Pelletier 1c33d6ce20 tomltest: custom comparison functions (#637) 2021-10-21 22:29:04 -04:00
Thomas Pelletier 3000471a12 parser: improve floats validation (#636) 2021-10-20 08:49:28 -04:00
Thomas Pelletier 1f33a6a476 tomltest: enable Valid_Datetime_LocalTime 2021-10-19 21:20:45 -04:00
Thomas Pelletier 2700aad5d2 tomltest: run UTF8 tests (#634) 2021-10-19 16:00:56 -04:00
Thomas Pelletier 7ccaa2744e tomltest: unmarshal JSONs for tests (#633) 
Comparing the output and the expected results byte-wise means we get false
negative when order doesn't matter (for example the ValidTableKeyword test).
 2021-10-19 15:29:49 -04:00
Johanan Idicula df4bb061f8 time: follow RFC3339 spec for datetime (#632) 2021-10-18 09:56:07 -04:00
Thomas Pelletier 9e81ce1c33 Create SECURITY.md 2021-10-17 20:57:34 -04:00
Cameron Moore a23850f29b decode: preserve nanosecond precision when decoding time (#626) 
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes>
 2021-10-17 20:43:29 -04:00
Johanan Idicula 76f53c857b unmarshal: validate date (#622) 2021-10-17 20:18:20 -04:00
Thomas Pelletier 85f5d567e4 parser: validate invalid ASCII control characters 2021-10-16 07:41:12 -04:00
Thomas Pelletier bd5cba0b0b Update benchmarks readme (#630) 
* Fix ci.sh for new benchmarks

Nice + taskset are more stable on my machine. We want to excude non high-level
benchmarks. BurntSushi/toml now supports canada.toml.

* Update latest benchmarks in README
 2021-10-15 19:53:40 -04:00
Thomas Pelletier cd54472d03 Validate UTF-8 (#629) 2021-10-15 19:13:21 -04:00
Thomas Pelletier cc0d1a90ff testgen: skip currently failing tests (#627) 2021-10-14 11:14:44 -04:00
Sterling Hanenkamp 4984dcb5e9 encode: ensure floats have decimal point (#615) 
Fixes #571

Co-authored-by: Sterling Hanenkamp <sterling@ziprecruiter.com>
 2021-10-14 08:34:54 -04:00
jidicula 86632bc190 parser: fail when missing array separator (#616) 
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes>
 2021-10-14 08:26:29 -04:00
Cameron Moore d25eec183f gotoml-test-decoder: add toml-test decoder command (#619) 2021-10-14 08:14:34 -04:00
Riya John e96746311c decoder: fix panic date time should have a timezone (#614) 
Fixes #596

Co-authored-by: Thomas Pelletier <thomas@pelletier.codes>
 2021-10-06 21:24:25 -04:00
Cameron Moore 62acca2b68 tomltestgen: add toml-test unit test generation command (#610) 
Tests are hidden behind a "testsuite" build tag for now since many tests
are failing.  Use `go test -tags testsuite` to activate.

Use `go generate` to regenerate toml_testgen_test.go.

Co-authored-by: Thomas Pelletier <thomas@pelletier.codes>
 2021-10-03 22:15:30 -04:00
Cameron Moore 476492a85c unmarshal: support lowercase 'T' and 'Z' in date-time parsing (#601) 
RFC3399 allows for lowercase 't' and 'z' in date-time values.

Fixes #600
 2021-09-25 10:02:23 -07:00
Thomas Pelletier ee9b902222 unmarshal: convert ints if target type is compatible (#594) 
This is required to support custom types.

Fixes #590
 2021-09-09 21:25:14 -04:00
Thomas Pelletier fa56f48daf parser: don't overflow when parsing bad times (#593) 
Fixes #585
 2021-09-09 11:59:37 -04:00
Thomas Pelletier f34c9c332f scanner: fix error reporting for last comments (#591) 
When an invalid TOML expression ends with a comment before the end of
file, the decode error would take a nil from scanComment, which is not
part of the document.

Fixes #588
 2021-09-08 21:54:30 -04:00
Thomas Pelletier a0d685d482 unmarshal: don't crash on unterminated inline table (#587) 
Fixes #586
 2021-09-07 20:08:59 -04:00
Thomas Pelletier 4a5ae9e81e errors: fix context generation with only one line 2021-09-07 10:36:22 -04:00
Thomas Pelletier 7e2fa1bc80 unmarshal: fix non-terminated array error 
Fixes #581
 2021-09-07 10:36:22 -04:00
Thomas Pelletier 40cfb6f458 parser: don't crash on unterminated table key (#580) 
* parser: don't crash on unterminated table key

Fixes #579

* parser: fix format of error returned by expect

EOF was missing the format string and %U is not very human friendly.
 2021-09-06 12:18:45 -04:00
Thomas Pelletier 1230ca485e unmarshal: make copy of non addressable values (#576) 
When unmarshaling into a nested struct in a map, the value is not
addressable. In that case, make a copy of it and modify it instead.

Fixes #575
 2021-08-31 20:22:38 -04:00
Thomas Pelletier 69ab7e10d1 Go 1.17 release (#574) 
Minimum supported version: Go 1.16.
 2021-08-17 09:43:52 -04:00
Thomas Pelletier fa07960695 Add installation instructions (#572) 2021-07-27 18:12:44 -04:00
kkHAIKE 8be357dfa1 Add LocalTime to interface{} decode support (#567) 
Co-authored-by: Thomas Pelletier <thomas@pelletier.codes>
 2021-07-21 17:50:12 +02:00
kkHAIKE a93b34d984 Unicode parsing optimization (#568) 
Inline call to hexToRune and uses specialized parsing, as found in encoding/json.

Co-authored-by: Thomas Pelletier <thomas@pelletier.codes>
 2021-07-21 10:50:03 +02:00
Matthieu MOREL 9c24fbeaad Set up Dependabot for GitHub actions and docker (#570) 2021-07-20 16:54:26 +02:00
Thomas Pelletier f6b38c33b7 Provide own implementation of Local* (#558) 
* Reduces the public API.
* Reuses optimized parsing functions.
* Removes reliance on Google code under Apache license.
 2021-06-08 20:27:05 -04:00
Thomas Pelletier 773f10110c Unmarshal recursive structs (#557) 
Co-authored-by: Nabetani <takenori@nabetani.sakura.ne.jp>
 2021-06-08 14:22:39 -04:00
Thomas Pelletier 618f0181ac AST Tweaks (#551) 
* Use pointers instead of copying around ast.Node

Node is a 56B struct that is constantly in the hot path. Passing nodes
around by copy had a cost that started to add up. This change replaces
them by pointers. Using unsafe pointer arithmetic and converting
sibling/child indexes to relative offsets, it removes the need to carry
around a pointer to the root of the tree. This saves 8B per Node. This
space will be used to store an extra []byte slice to provide contextual
error handling on all nodes, including the ones whose data is different
than the raw input (for example: strings with escaped characters), while
staying under the size of a cache line.

* Remove conditional

* Add Raw to track range in data for parsed values

* Simplify reference tracking
v2.0.0-beta.3 		2021-06-03 21:48:51 -04:00
Thomas Pelletier f3bb20ea79 Benchmark marshal (#550) 2021-06-02 09:29:19 -04:00
Thomas Pelletier b0d6c62255 Don't use bytes.Buffer when not necessary (#549) 
When parsing strings, they can be referenced directly from the document
when they don't contain escaped characters. This avoids paying to cost
of allocating (and sometimes growing) the bytes buffer unecessarily.
 2021-06-01 09:51:59 -04:00
Thomas Pelletier b202375414 Add benchmarks results to readme (#548) 2021-06-01 09:10:17 -04:00
Thomas Pelletier 250e073408 Stack-based unmarshaler (#546) 
* Benchmark script

* Rewrite unmarshaler using the stack

Instead of tracking the build chain using `target`s, use the stack
instead.

Working and most benchmarks look good, but regression on structs unmarshalling.

~60% slower on ReferenceFile/struct.

* Shortcut to check if last node of iterator

* Remove unecessary pointer allocation

* Skip over unused keys without marking them as seen

* Add some tests

* Fix mktemp on macos
 2021-05-31 12:14:13 -04:00
Thomas Pelletier 11f022ab09 Fix parser skipping over the whole file (#547) 
* Add test for ReferenceFile/struct
* Stop skipping after table
* Add .gitattributes to force LF encoding
* Fix the reference file
 2021-05-30 18:53:07 -04:00
Thomas Pelletier 840df4a229 seen tracker: use array for storage (#545) 
name                              old time/op    new time/op    delta
UnmarshalDataset/config-32          81.2ms ± 3%    77.8ms ± 3%   -4.25%  (p=0.000 n=20+19)
UnmarshalDataset/canada-32           104ms ± 5%     105ms ± 4%     ~     (p=0.102 n=20+20)
UnmarshalDataset/citm_catalog-32    57.5ms ± 5%    59.0ms ± 5%   +2.54%  (p=0.033 n=20+20)
UnmarshalDataset/twitter-32         25.7ms ± 7%    28.1ms ± 5%   +9.33%  (p=0.000 n=20+20)
UnmarshalDataset/code-32             305ms ± 6%     292ms ± 5%   -4.29%  (p=0.000 n=20+19)
UnmarshalDataset/example-32          519µs ± 6%     522µs ± 5%     ~     (p=0.659 n=20+20)
UnmarshalSimple/struct-32           1.44µs ± 1%    1.17µs ± 6%  -18.78%  (p=0.000 n=14+20)
UnmarshalSimple/map-32              2.30µs ± 4%    1.99µs ± 4%  -13.65%  (p=0.000 n=20+19)
ReferenceFile/struct-32             44.1µs ± 4%    38.1µs ± 5%  -13.61%  (p=0.000 n=18+20)
ReferenceFile/map-32                 197µs ± 7%     189µs ± 5%   -3.91%  (p=0.000 n=20+20)
HugoFrontMatter-32                  45.9µs ± 6%    39.3µs ± 6%  -14.46%  (p=0.000 n=19+20)

name                              old speed      new speed      delta
UnmarshalDataset/config-32        12.9MB/s ± 3%  13.5MB/s ± 3%   +4.42%  (p=0.000 n=20+19)
UnmarshalDataset/canada-32        21.1MB/s ± 5%  20.9MB/s ± 4%     ~     (p=0.101 n=20+20)
UnmarshalDataset/citm_catalog-32  9.72MB/s ± 6%  9.47MB/s ± 5%   -2.53%  (p=0.031 n=20+20)
UnmarshalDataset/twitter-32       17.2MB/s ± 7%  15.8MB/s ± 5%   -8.57%  (p=0.000 n=20+20)
UnmarshalDataset/code-32          8.81MB/s ± 7%  9.20MB/s ± 5%   +4.47%  (p=0.000 n=20+19)
UnmarshalDataset/example-32       15.6MB/s ± 6%  15.5MB/s ± 5%     ~     (p=0.644 n=20+20)
UnmarshalSimple/struct-32         7.61MB/s ± 1%  9.39MB/s ± 7%  +23.33%  (p=0.000 n=15+20)
UnmarshalSimple/map-32            4.78MB/s ± 4%  5.54MB/s ± 5%  +15.85%  (p=0.000 n=20+19)
ReferenceFile/struct-32            119MB/s ± 4%   138MB/s ± 5%  +15.79%  (p=0.000 n=18+20)
ReferenceFile/map-32              26.6MB/s ± 7%  27.7MB/s ± 5%   +4.06%  (p=0.000 n=20+20)
HugoFrontMatter-32                11.9MB/s ± 6%  13.9MB/s ± 6%  +16.91%  (p=0.000 n=19+20)

name                              old alloc/op   new alloc/op   delta
UnmarshalDataset/config-32          16.9MB ± 0%    13.9MB ± 0%  -17.48%  (p=0.000 n=18+18)
UnmarshalDataset/canada-32          74.3MB ± 0%    74.3MB ± 0%   -0.00%  (p=0.000 n=20+20)
UnmarshalDataset/citm_catalog-32    37.3MB ± 0%    37.3MB ± 0%   +0.11%  (p=0.000 n=20+20)
UnmarshalDataset/twitter-32         15.6MB ± 0%    15.6MB ± 0%     ~     (p=0.211 n=19+20)
UnmarshalDataset/code-32            59.5MB ± 0%    52.4MB ± 0%  -11.96%  (p=0.000 n=19+20)
UnmarshalDataset/example-32          238kB ± 0%     239kB ± 0%   +0.02%  (p=0.000 n=18+20)
UnmarshalSimple/struct-32             981B ± 0%      709B ± 0%  -27.73%  (p=0.000 n=20+20)
UnmarshalSimple/map-32              1.45kB ± 0%    1.17kB ± 0%  -18.82%  (p=0.000 n=20+20)
ReferenceFile/struct-32             11.8kB ± 0%     9.7kB ± 0%  -17.64%  (p=0.000 n=20+20)
ReferenceFile/map-32                51.5kB ± 0%    52.2kB ± 0%   +1.30%  (p=0.000 n=20+17)
HugoFrontMatter-32                  12.1kB ± 0%    11.1kB ± 0%   -7.97%  (p=0.000 n=20+19)

name                              old allocs/op  new allocs/op  delta
UnmarshalDataset/config-32            645k ± 0%      557k ± 0%  -13.76%  (p=0.000 n=20+16)
UnmarshalDataset/canada-32            896k ± 0%      896k ± 0%   -0.00%  (p=0.000 n=20+20)
UnmarshalDataset/citm_catalog-32      380k ± 0%      377k ± 0%   -0.75%  (p=0.000 n=19+20)
UnmarshalDataset/twitter-32           158k ± 0%      158k ± 0%   -0.01%  (p=0.000 n=18+18)
UnmarshalDataset/code-32             2.92M ± 0%     2.57M ± 0%  -11.87%  (p=0.000 n=20+20)
UnmarshalDataset/example-32          3.66k ± 0%     3.64k ± 0%   -0.63%  (p=0.000 n=20+20)
UnmarshalSimple/struct-32             13.0 ± 0%      10.0 ± 0%  -23.08%  (p=0.000 n=20+20)
UnmarshalSimple/map-32                22.0 ± 0%      19.0 ± 0%  -13.64%  (p=0.000 n=20+20)
ReferenceFile/struct-32                253 ± 0%       155 ± 0%  -38.74%  (p=0.000 n=20+20)
ReferenceFile/map-32                 1.67k ± 0%     1.44k ± 0%  -14.23%  (p=0.000 n=20+20)
HugoFrontMatter-32                     357 ± 0%       313 ± 0%  -12.32%  (p=0.000 n=20+20)
 2021-05-26 18:47:00 -04:00
Thomas Pelletier c2d1fd86e5 Fix timezone detection when time has fractional component (#544) 2021-05-21 09:37:43 -04:00
Thomas Pelletier 238a6fef7d Add links to proper feedback channels 2021-05-15 08:55:07 -04:00
Thomas Pelletier 67852cf007 Clarify default struct tag being unsupported (#543) 
Follow up to https://github.com/pelletier/go-toml/issues/542
 2021-05-15 08:49:15 -04:00
Thomas Pelletier d276c42adc Run coverage test on branches only 2021-05-10 20:22:12 -04:00
Thomas Pelletier 95c701b253 Increase test coverage (#538) 
Also fix array in map bug.
v2.0.0-beta.2 		2021-05-10 20:17:05 -04:00
Thomas Pelletier 3db329a512 ci: basic github action for coverage (#537) 2021-05-09 17:37:03 -04:00
Thomas Pelletier 45ea20024b Readme (#535) v2.0.0-beta.1 2021-05-08 17:03:51 -04:00
Thomas Pelletier ea225df3ed v2: errors (#534) 
```
name                              old time/op    new time/op    delta
UnmarshalDataset/config-32          86.7ms ± 2%    87.5ms ± 2%     ~     (p=0.113 n=9+10)
UnmarshalDataset/canada-32           129ms ± 4%     106ms ± 3%  -17.94%  (p=0.000 n=10+10)
UnmarshalDataset/citm_catalog-32    59.4ms ± 5%    58.7ms ± 5%     ~     (p=0.393 n=10+10)
UnmarshalDataset/twitter-32         27.0ms ± 7%    26.9ms ± 6%     ~     (p=0.720 n=10+9)
UnmarshalDataset/code-32             326ms ± 4%     322ms ± 7%     ~     (p=0.661 n=9+10)
UnmarshalDataset/example-32          510µs ±11%     526µs ± 7%     ~     (p=0.182 n=10+9)
UnmarshalSimple-32                  1.41µs ± 6%    1.41µs ± 4%     ~     (p=0.736 n=10+9)
ReferenceFile-32                    45.6µs ± 3%    43.9µs ±10%     ~     (p=0.089 n=10+10)

name                              old speed      new speed      delta
UnmarshalDataset/config-32        12.1MB/s ± 2%  12.0MB/s ± 2%     ~     (p=0.108 n=9+10)
UnmarshalDataset/canada-32        17.1MB/s ± 4%  20.9MB/s ± 3%  +21.86%  (p=0.000 n=10+10)
UnmarshalDataset/citm_catalog-32  9.41MB/s ± 5%  9.51MB/s ± 5%     ~     (p=0.362 n=10+10)
UnmarshalDataset/twitter-32       16.4MB/s ± 8%  16.5MB/s ± 6%     ~     (p=0.704 n=10+9)
UnmarshalDataset/code-32          8.24MB/s ± 4%  8.34MB/s ± 7%     ~     (p=0.675 n=9+10)
UnmarshalDataset/example-32       15.9MB/s ±11%  15.4MB/s ± 7%     ~     (p=0.182 n=10+9)
ReferenceFile-32                   115MB/s ± 4%   120MB/s ±10%     ~     (p=0.085 n=10+10)

name                              old alloc/op   new alloc/op   delta
UnmarshalDataset/config-32          16.9MB ± 0%    16.9MB ± 0%   -0.02%  (p=0.000 n=10+10)
UnmarshalDataset/canada-32          76.8MB ± 0%    74.3MB ± 0%   -3.31%  (p=0.000 n=10+10)
UnmarshalDataset/citm_catalog-32    37.3MB ± 0%    37.1MB ± 0%   -0.60%  (p=0.000 n=9+10)
UnmarshalDataset/twitter-32         15.6MB ± 0%    15.6MB ± 0%   -0.09%  (p=0.000 n=10+10)
UnmarshalDataset/code-32            60.2MB ± 0%    59.3MB ± 0%   -1.51%  (p=0.000 n=10+9)
UnmarshalDataset/example-32          238kB ± 0%     238kB ± 0%   -0.18%  (p=0.000 n=10+10)
ReferenceFile-32                    11.8kB ± 0%    11.8kB ± 0%     ~     (all equal)

name                              old allocs/op  new allocs/op  delta
UnmarshalDataset/config-32            653k ± 0%      645k ± 0%   -1.20%  (p=0.000 n=10+6)
UnmarshalDataset/canada-32           1.01M ± 0%     0.90M ± 0%  -11.04%  (p=0.000 n=9+10)
UnmarshalDataset/citm_catalog-32      384k ± 0%      370k ± 0%   -3.75%  (p=0.000 n=10+10)
UnmarshalDataset/twitter-32           160k ± 0%      157k ± 0%   -1.32%  (p=0.000 n=10+10)
UnmarshalDataset/code-32             2.97M ± 0%     2.91M ± 0%   -2.15%  (p=0.000 n=10+7)
UnmarshalDataset/example-32          3.69k ± 0%     3.63k ± 0%   -1.52%  (p=0.000 n=10+10)
ReferenceFile-32                       253 ± 0%       253 ± 0%     ~     (all equal)
```
 2021-05-08 16:04:25 -04:00
Thomas Pelletier 4545a3e94b ci: remove benchmarks 
Both github actions and my own VPS have too much noise to be useful.
 2021-05-07 23:34:17 -04:00