Compare commits

...

29 Commits

Author SHA1 Message Date
Thomas Pelletier 903d9455db TOML 0.5.0 (#320)
go-toml now officially supports all TOML 0.5.0 features. If anything
does not work according to the spec, please file a bug!
2019-10-25 14:53:56 -04:00
Thomas Pelletier a89a075e1b Test for accidental newlines (#319) 2019-10-25 14:44:53 -04:00
Thomas Pelletier 5e74bb91ea Local time support (#318) 2019-10-25 14:28:32 -04:00
Thomas Pelletier 3a4d7af89e Local DateTime support (#317) 2019-10-25 14:07:46 -04:00
Thomas Pelletier 8a362ad712 Short-date support (#298) 2019-10-25 13:21:44 -04:00
Thomas Pelletier 5edf9acd3e Add testing for encodeMultilineTomlString (#313) 2019-10-21 14:31:28 -04:00
Thomas Pelletier e95df67ba3 Delete token.Int() (#312)
Not used anywhere.
2019-10-21 14:01:10 -04:00
Jonathan Lloyd bef0f57967 Fix key parsing in line tables (#311)
A bug was reported that indicated that inline tables did not fully support bare keys:
$ echo 'foo = { -bar => "buz"}' | ./tomljson
(1, 9): unexpected token type in inline table: Error

$ echo 'foo = { "whatever" = "buz"}' | ./tomljson
(1, 10): unexpected token type in inline table: String

echo 'foo = { _no = "buz"}' | ./tomljson
(1, 9): unexpected token type in inline table: Error

This change makes a couple of tweaks to to allow for all key variants in inline tables

Fixes: #282
2019-10-20 20:36:14 -04:00
Marcin Białoń e87c92d4f4 Marshal arrays (#310)
Fixes #285
2019-10-09 12:33:56 -04:00
dependabot-preview[bot] 8fe62057ea Bump gopkg.in/yaml.v2 from 2.2.3 to 2.2.4 (#309)
Bumps [gopkg.in/yaml.v2](https://github.com/go-yaml/yaml) from 2.2.3 to 2.2.4.
- [Release notes](https://github.com/go-yaml/yaml/releases)
- [Commits](https://github.com/go-yaml/yaml/compare/v2.2.3...v2.2.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-05 15:18:21 -04:00
dependabot-preview[bot] 5f42261979 Bump gopkg.in/yaml.v2 from 2.2.2 to 2.2.3 (#308)
Bumps [gopkg.in/yaml.v2](https://github.com/go-yaml/yaml) from 2.2.2 to 2.2.3.
- [Release notes](https://github.com/go-yaml/yaml/releases)
- [Commits](https://github.com/go-yaml/yaml/compare/v2.2.2...v2.2.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-01 06:57:01 -04:00
Marcin Białoń 75654e60b8 Tree.Marshal returns the TOML encoding of Tree (#306)
The Tree.Marshal tried to marshal the Tree struct itself rather than the nodes being part of the tree.

Fixes #295
2019-09-26 13:39:15 -07:00
Thomas Pelletier 091e2dc498 Set up CI with Azure Pipelines (#304) 2019-09-24 16:11:17 -07:00
Marcin Białoń 095a905e04 Allow space to separate date and time (#300)
Fixes #231
2019-09-19 10:45:53 -07:00
Thomas Pelletier ec312409d3 Update fuzzit.dev script to latest (#301)
Fixes #299
2019-09-18 09:04:23 -07:00
Thomas Pelletier 26fd12ff54 Fix fuzzit master (#297) 2019-09-09 20:59:59 -07:00
Thomas Pelletier b40204d36a Replace CIs by Github Actions (#294) 2019-09-09 19:44:45 -07:00
Gaurav Dhameeja 4d5afd743f jsontoml tool (#296)
`jsontoml` is very similar to `tomljson`
It uses json.Unmarshal to convert read json to map and then
converts the map to tree using `toml.TreeFromMap`.
Then this tree is converted to toml using `tree.toTomlString()`
The numbers when taken as input from json get converted to float64
because of how `json.Unmarshal()` converts all json numbers to float.

Fixes #280
2019-09-06 09:36:56 -07:00
Chris 3ded2e09ee Fix float64 truncation error (#293)
Don't truncate float64 representation on marashaling.

Fixes https://github.com/pelletier/go-toml/issues/289
2019-08-26 20:57:02 -07:00
Yevgeny Pats 781fbae71e Add fuzzit.dev continuous fuzzing integration (#288) 2019-08-19 12:53:00 -07:00
Thomas Pelletier 68063a447e Quote keys during encoding when the key isn't bare (#291)
In case the key contains non-bare characters (out of `A-Za-z0-9_-`), the
key needs to be quoted during encoding to be valid TOML.
2019-08-18 23:00:12 -07:00
Roberth Kulbin 84da2c4a25 Merge struct fields in Unmarshal (#284)
* add test for unexported field preservation
* merge struct values instead of replacing them
* use struct merging on nested value structs
* unmarshalling merges nested struct pointers when non-nil
2019-07-25 00:06:17 -07:00
Kamil Samigullin dba45d427f Handle anonymous structs (#281)
Handle anonymous structs during Unmarshal.

Fixes #279
2019-05-29 20:55:49 -07:00
Gregory Oschwald 728039f679 Handle other key types in Unmarshal (#276)
Previously, this would fail with:

```
panic: reflect.Value.SetMapIndex: value of type string is not assignable to type toml.letter [recovered]
panic: reflect.Value.SetMapIndex: value of type string is not assignable to type toml.letter
```

Now this only panics when the key type cannot be converted from a
string.
2019-04-29 20:50:10 -07:00
Gregory Oschwald 1d8903f1d0 Allow unmarshaling to top level maps (#273) 2019-04-24 23:15:40 -07:00
Brent DeSpain 65b27e6823 Order map keys alphabetically (#270)
This makes sure we have a stable output when marshaling
maps.

Fixes #268
2019-04-11 13:52:29 +01:00
Thomas Pelletier 6ea91ef590 Do not push Docker images for forked repositories (#272)
For security reasons, CircleCI does not make environment variables
available on forked repositories (often used in PRs). This will still
build the docker image, but won't try to push it to dockerhub.
2019-04-11 13:49:07 +01:00
Ceriath 51edd0ca49 Fix goreportcard issues (#271)
* Fixed misspell

* Fixed ineffassign

`user` and `password` always got overwritten
`orderedVals` was initialized with an empty array but always got overwritten by either `sortByLines()` or `sortAlphabetical`
`err` was assigned a `nil` value that was either overwritten or unused anyways

* Fix comment for DeletePath

The comment assumed the method was named Delete, i guess a rename happened at some point

* Update doc_test.go
2019-04-11 12:11:29 +01:00
Thomas Pelletier d95bfe020e Dockerfile (#269)
Provide docker images for go-toml tools.

Ref: https://github.com/pelletier/go-toml/pull/267
2019-04-10 13:43:12 +01:00
32 changed files with 2524 additions and 321 deletions
-140
View File
@@ -1,140 +0,0 @@
version: 2.1
executors:
golang:
parameters:
version:
type: string
docker:
- image: circleci/golang:<< parameters.version >>
commands:
get_deps:
description: "Get go dependencies"
steps:
- run: go get github.com/jstemmer/go-junit-report
run_test:
description: "Run unit tests for a go module"
parameters:
test_name:
type: string
module:
type: string
coverage:
default: false
type: boolean
allow_fail:
type: boolean
default: false
steps:
- run:
name: "Run tests for <<parameters.test_name>>"
command: |
TEST_DIR="/tmp/test-results/<<parameters.test_name>>"
mkdir -p ${TEST_DIR}
trap "go-junit-report </tmp/test-results/go-test.out > ${TEST_DIR}/go-test-report.xml" EXIT
go test <<parameters.module>> -race -v \
<<# parameters.coverage >>-coverprofile=/tmp/workspace/coverage.txt -covermode=atomic<</ parameters.coverage >> \
| tee /tmp/test-results/go-test.out <<# parameters.allow_fail >>|| true<</ parameters.allow_fail >>
jobs:
go:
parameters:
version:
type: string
allow_fail:
type: boolean
default: false
executor:
name: golang
version: "<<parameters.version>>"
working_directory: /go/src/github.com/pelletier/go-toml
environment:
GO111MODULE: "on"
steps:
- checkout
- run: mkdir -p /tmp/workspace
- run: go fmt ./... <<# parameters.allow_fail >>|| true<</ parameters.allow_fail >>
- get_deps
- run_test:
test_name: "go-toml"
module: "github.com/pelletier/go-toml"
coverage: true
allow_fail: <<parameters.allow_fail>>
- run_test:
test_name: "tomljson"
module: "github.com/pelletier/go-toml/cmd/tomljson"
allow_fail: <<parameters.allow_fail>>
- run_test:
test_name: "tomll"
module: "github.com/pelletier/go-toml/cmd/tomll"
allow_fail: <<parameters.allow_fail>>
- run_test:
test_name: "query"
module: "github.com/pelletier/go-toml/query"
allow_fail: <<parameters.allow_fail>>
- store_test_results:
path: /tmp/test-results
codecov:
docker:
- image: "circleci/golang:1.12"
steps:
- attach_workspace:
at: /tmp/workspace
- run:
name: "upload to codecov"
working_directory: /tmp/workspace
command: |
curl https://codecov.io/bash > codecov.sh
bash codecov.sh -v
workflows:
version: 2.1
build:
jobs:
- go:
name: "go1_11"
version: "1.11"
- go:
name: "go1_12"
version: "1.12"
post-steps:
- run: go tool cover -html=/tmp/workspace/coverage.txt -o coverage.html
- store_artifacts:
path: /tmp/workspace/coverage.txt
- store_artifacts:
path: coverage.html
- persist_to_workspace:
root: /tmp/workspace
paths:
- coverage.txt
- go:
name: "gotip"
version: "1.12" # use as base
allow_fail: true
pre-steps:
- restore_cache:
keys:
- go-tip-source
- run:
name: "Compile go tip"
command: |
if [ ! -d "/tmp/go" ]; then
git clone https://go.googlesource.com/go /tmp/go
fi
cd /tmp/go
git checkout master
git pull
cd src
./make.bash
echo 'export PATH="/tmp/go/bin:$PATH"' >> $BASH_ENV
- run: go version
- save_cache:
key: go-tip-source
paths:
- "/tmp/go"
- codecov:
requires:
- go1_11
- go1_12
+2
View File
@@ -0,0 +1,2 @@
cmd/tomll/tomll
cmd/tomljson/tomljson
+3
View File
@@ -1,2 +1,5 @@
test_program/test_program_bin
fuzz/
cmd/tomll/tomll
cmd/tomljson/tomljson
cmd/tomltestgen/tomltestgen
-22
View File
@@ -1,22 +0,0 @@
sudo: false
language: go
go:
- 1.11.x
- 1.12.x
- tip
matrix:
allow_failures:
- go: tip
fast_finish: true
env:
- GO111MODULE=on
script:
- if [ -n "$(go fmt ./...)" ]; then exit 1; fi
- go test github.com/pelletier/go-toml -race -coverprofile=coverage.txt -covermode=atomic
- go test github.com/pelletier/go-toml/cmd/tomljson
- go test github.com/pelletier/go-toml/cmd/tomll
- go test github.com/pelletier/go-toml/query
- ./benchmark.sh $TRAVIS_BRANCH https://github.com/$TRAVIS_REPO_SLUG.git
after_success:
- bash <(curl -s https://codecov.io/bash)
+11
View File
@@ -0,0 +1,11 @@
FROM golang:1.12-alpine3.9 as builder
WORKDIR /go/src/github.com/pelletier/go-toml
COPY . .
ENV CGO_ENABLED=0
ENV GOOS=linux
RUN go install ./...
FROM scratch
COPY --from=builder /go/bin/tomll /usr/bin/tomll
COPY --from=builder /go/bin/tomljson /usr/bin/tomljson
COPY --from=builder /go/bin/jsontoml /usr/bin/jsontoml
+26 -3
View File
@@ -3,12 +3,11 @@
Go library for the [TOML](https://github.com/mojombo/toml) format.
This library supports TOML version
[v0.4.0](https://github.com/toml-lang/toml/blob/master/versions/en/toml-v0.4.0.md)
[v0.5.0](https://github.com/toml-lang/toml/blob/master/versions/en/toml-v0.5.0.md)
[![GoDoc](https://godoc.org/github.com/pelletier/go-toml?status.svg)](http://godoc.org/github.com/pelletier/go-toml)
[![license](https://img.shields.io/github/license/pelletier/go-toml.svg)](https://github.com/pelletier/go-toml/blob/master/LICENSE)
[![Build Status](https://travis-ci.org/pelletier/go-toml.svg?branch=master)](https://travis-ci.org/pelletier/go-toml)
[![Windows Build status](https://ci.appveyor.com/api/projects/status/4aepwwjori266hkt/branch/master?svg=true)](https://ci.appveyor.com/project/pelletier/go-toml/branch/master)
[![Build Status](https://dev.azure.com/pelletierthomas/go-toml-ci/_apis/build/status/pelletier.go-toml?branchName=master)](https://dev.azure.com/pelletierthomas/go-toml-ci/_build/latest?definitionId=1&branchName=master)
[![codecov](https://codecov.io/gh/pelletier/go-toml/branch/master/graph/badge.svg)](https://codecov.io/gh/pelletier/go-toml)
[![Go Report Card](https://goreportcard.com/badge/github.com/pelletier/go-toml)](https://goreportcard.com/report/github.com/pelletier/go-toml)
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fpelletier%2Fgo-toml.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2Fpelletier%2Fgo-toml?ref=badge_shield)
@@ -101,6 +100,30 @@ Go-toml provides two handy command line tools:
tomljson --help
```
* `jsontoml`: Reads a JSON file and outputs a TOML representation.
```
go install github.com/pelletier/go-toml/cmd/jsontoml
jsontoml --help
```
### Docker image
Those tools are also availble as a Docker image from
[dockerhub](https://hub.docker.com/r/pelletier/go-toml). For example, to
use `tomljson`:
```
docker run -v $PWD:/workdir pelletier/go-toml tomljson /workdir/example.toml
```
Only master (`latest`) and tagged versions are published to dockerhub. You
can build your own image as usual:
```
docker build -t go-toml .
```
## Contribute
Feel free to report bugs and patches using GitHub's pull requests system on
-34
View File
@@ -1,34 +0,0 @@
version: "{build}"
# Source Config
clone_folder: c:\gopath\src\github.com\pelletier\go-toml
# Build host
environment:
GOPATH: c:\gopath
DEPTESTBYPASS501: 1
GOVERSION: 1.12
GO111MODULE: on
init:
- git config --global core.autocrlf input
# Build
install:
# Install the specific Go version.
- rmdir c:\go /s /q
- appveyor DownloadFile https://storage.googleapis.com/golang/go%GOVERSION%.windows-amd64.msi
- msiexec /i go%GOVERSION%.windows-amd64.msi /q
- choco install bzr
- set Path=c:\go\bin;c:\gopath\bin;C:\Program Files (x86)\Bazaar\;C:\Program Files\Mercurial\%Path%
- go version
- go env
build: false
deploy: false
test_script:
- go test github.com/pelletier/go-toml
- go test github.com/pelletier/go-toml/cmd/tomljson
- go test github.com/pelletier/go-toml/cmd/tomll
- go test github.com/pelletier/go-toml/query
+167
View File
@@ -0,0 +1,167 @@
trigger:
- master
stages:
- stage: fuzzit
displayName: "Run Fuzzit"
dependsOn: []
condition: and(succeeded(), eq(variables['Build.SourceBranchName'], 'master'))
jobs:
- job: submit
displayName: "Submit"
pool:
vmImage: ubuntu-latest
steps:
- task: GoTool@0
displayName: "Install Go 1.13"
inputs:
version: "1.13"
- script: echo "##vso[task.setvariable variable=PATH]${PATH}:/home/vsts/go/bin/"
- script: mkdir -p ${HOME}/go/src/github.com/pelletier/go-toml
- script: cp -R . ${HOME}/go/src/github.com/pelletier/go-toml
- task: Bash@3
inputs:
filePath: './fuzzit.sh'
env:
TYPE: fuzzing
FUZZIT_API_KEY: $(FUZZIT_API_KEY)
- stage: run_checks
displayName: "Check"
dependsOn: []
jobs:
- job: fmt
displayName: "fmt"
pool:
vmImage: ubuntu-latest
steps:
- task: GoTool@0
displayName: "Install Go 1.13"
inputs:
version: "1.13"
- task: Go@0
displayName: "go fmt ./..."
inputs:
command: 'custom'
customCommand: 'fmt'
arguments: './...'
- job: coverage
displayName: "coverage"
pool:
vmImage: ubuntu-latest
steps:
- task: GoTool@0
displayName: "Install Go 1.13"
inputs:
version: "1.13"
- task: Go@0
displayName: "Generate coverage"
inputs:
command: 'test'
arguments: "-race -coverprofile=coverage.txt -covermode=atomic"
- task: Bash@3
inputs:
targetType: 'inline'
script: 'bash <(curl -s https://codecov.io/bash) -t $(CODECOV_TOKEN)'
- job: benchmark
displayName: "benchmark"
pool:
vmImage: ubuntu-latest
steps:
- task: GoTool@0
displayName: "Install Go 1.13"
inputs:
version: "1.13"
- script: echo "##vso[task.setvariable variable=PATH]${PATH}:/home/vsts/go/bin/"
- task: Bash@3
inputs:
filePath: './benchmark.sh'
arguments: "master $(Build.Repository.Uri)"
- job: fuzzing
displayName: "fuzzing"
pool:
vmImage: ubuntu-latest
steps:
- task: GoTool@0
displayName: "Install Go 1.13"
inputs:
version: "1.13"
- script: echo "##vso[task.setvariable variable=PATH]${PATH}:/home/vsts/go/bin/"
- script: mkdir -p ${HOME}/go/src/github.com/pelletier/go-toml
- script: cp -R . ${HOME}/go/src/github.com/pelletier/go-toml
- task: Bash@3
inputs:
filePath: './fuzzit.sh'
env:
TYPE: local-regression
- job: go_unit_tests
displayName: "unit tests"
strategy:
matrix:
linux 1.13:
goVersion: '1.13'
imageName: 'ubuntu-latest'
mac 1.13:
goVersion: '1.13'
imageName: 'macos-10.13'
windows 1.13:
goVersion: '1.13'
imageName: 'vs2017-win2016'
linux 1.12:
goVersion: '1.12'
imageName: 'ubuntu-latest'
mac 1.12:
goVersion: '1.12'
imageName: 'macos-10.13'
windows 1.12:
goVersion: '1.12'
imageName: 'vs2017-win2016'
pool:
vmImage: $(imageName)
steps:
- task: GoTool@0
displayName: "Install Go $(goVersion)"
inputs:
version: $(goVersion)
- task: Go@0
displayName: "go test ./..."
inputs:
command: 'test'
arguments: './...'
- stage: build_docker_image
displayName: "Build Docker image"
dependsOn: run_checks
jobs:
- job: build
displayName: "Build"
pool:
vmImage: ubuntu-latest
steps:
- task: Docker@2
inputs:
command: 'build'
Dockerfile: 'Dockerfile'
buildContext: '.'
addPipelineData: false
- stage: publish_docker_image
displayName: "Publish Docker image"
dependsOn: build_docker_image
condition: and(succeeded(), eq(variables['Build.SourceBranchName'], 'master'))
jobs:
- job: publish
displayName: "Publish"
pool:
vmImage: ubuntu-latest
steps:
- task: Docker@2
inputs:
containerRegistry: 'DockerHub'
repository: 'pelletier/go-toml'
command: 'buildAndPush'
Dockerfile: 'Dockerfile'
buildContext: '.'
tags: 'latest'
+1 -2
View File
@@ -1,6 +1,6 @@
#!/bin/bash
set -e
set -ex
reference_ref=${1:-master}
reference_git=${2:-.}
@@ -8,7 +8,6 @@ reference_git=${2:-.}
if ! `hash benchstat 2>/dev/null`; then
echo "Installing benchstat"
go get golang.org/x/perf/cmd/benchstat
go install golang.org/x/perf/cmd/benchstat
fi
tempdir=`mktemp -d /tmp/go-toml-benchmark-XXXXXX`
+82
View File
@@ -0,0 +1,82 @@
// Jsontoml reads JSON and converts to TOML.
//
// Usage:
// cat file.toml | jsontoml > file.json
// jsontoml file1.toml > file.json
package main
import (
"encoding/json"
"flag"
"fmt"
"io"
"io/ioutil"
"os"
"github.com/pelletier/go-toml"
)
func main() {
flag.Usage = func() {
fmt.Fprintln(os.Stderr, "jsontoml can be used in two ways:")
fmt.Fprintln(os.Stderr, "Writing to STDIN and reading from STDOUT:")
fmt.Fprintln(os.Stderr, "")
fmt.Fprintln(os.Stderr, "")
fmt.Fprintln(os.Stderr, "Reading from a file name:")
fmt.Fprintln(os.Stderr, " tomljson file.toml")
}
flag.Parse()
os.Exit(processMain(flag.Args(), os.Stdin, os.Stdout, os.Stderr))
}
func processMain(files []string, defaultInput io.Reader, output io.Writer, errorOutput io.Writer) int {
// read from stdin and print to stdout
inputReader := defaultInput
if len(files) > 0 {
file, err := os.Open(files[0])
if err != nil {
printError(err, errorOutput)
return -1
}
inputReader = file
defer file.Close()
}
s, err := reader(inputReader)
if err != nil {
printError(err, errorOutput)
return -1
}
io.WriteString(output, s)
return 0
}
func printError(err error, output io.Writer) {
io.WriteString(output, err.Error()+"\n")
}
func reader(r io.Reader) (string, error) {
jsonMap := make(map[string]interface{})
jsonBytes, err := ioutil.ReadAll(r)
if err != nil {
return "", err
}
err = json.Unmarshal(jsonBytes, &jsonMap)
if err != nil {
return "", err
}
tree, err := toml.TreeFromMap(jsonMap)
if err != nil {
return "", err
}
return mapToTOML(tree)
}
func mapToTOML(t *toml.Tree) (string, error) {
tomlBytes, err := t.ToTomlString()
if err != nil {
return "", err
}
return string(tomlBytes[:]), nil
}
+92
View File
@@ -0,0 +1,92 @@
package main
import (
"bytes"
"io/ioutil"
"os"
"runtime"
"strings"
"testing"
)
func expectBufferEquality(t *testing.T, name string, buffer *bytes.Buffer, expected string) {
output := buffer.String()
if output != expected {
t.Errorf("incorrect %s: \n%sexpected %s: \n%s", name, output, name, expected)
t.Log([]rune(output))
t.Log([]rune(expected))
}
}
func expectProcessMainResults(t *testing.T, input string, args []string, exitCode int, expectedOutput string, expectedError string) {
inputReader := strings.NewReader(input)
outputBuffer := new(bytes.Buffer)
errorBuffer := new(bytes.Buffer)
returnCode := processMain(args, inputReader, outputBuffer, errorBuffer)
expectBufferEquality(t, "output", outputBuffer, expectedOutput)
expectBufferEquality(t, "error", errorBuffer, expectedError)
if returnCode != exitCode {
t.Error("incorrect return code:", returnCode, "expected", exitCode)
}
}
func TestProcessMainReadFromStdin(t *testing.T) {
expectedOutput := `
[mytoml]
a = 42.0
`
input := `{
"mytoml": {
"a": 42
}
}
`
expectedError := ``
expectedExitCode := 0
expectProcessMainResults(t, input, []string{}, expectedExitCode, expectedOutput, expectedError)
}
func TestProcessMainReadFromFile(t *testing.T) {
input := `{
"mytoml": {
"a": 42
}
}
`
tmpfile, err := ioutil.TempFile("", "example.json")
if err != nil {
t.Fatal(err)
}
if _, err := tmpfile.Write([]byte(input)); err != nil {
t.Fatal(err)
}
defer os.Remove(tmpfile.Name())
expectedOutput := `
[mytoml]
a = 42.0
`
expectedError := ``
expectedExitCode := 0
expectProcessMainResults(t, ``, []string{tmpfile.Name()}, expectedExitCode, expectedOutput, expectedError)
}
func TestProcessMainReadFromMissingFile(t *testing.T) {
var expectedError string
if runtime.GOOS == "windows" {
expectedError = `open /this/file/does/not/exist: The system cannot find the path specified.
`
} else {
expectedError = `open /this/file/does/not/exist: no such file or directory
`
}
expectProcessMainResults(t, ``, []string{"/this/file/does/not/exist"}, -1, ``, expectedError)
}
+1 -1
View File
@@ -1,7 +1,7 @@
// Package toml is a TOML parser and manipulation library.
//
// This version supports the specification as described in
// https://github.com/toml-lang/toml/blob/master/versions/en/toml-v0.4.0.md
// https://github.com/toml-lang/toml/blob/master/versions/en/toml-v0.5.0.md
//
// Marshaling
//
+5 -4
View File
@@ -16,13 +16,14 @@ func Example_tree() {
fmt.Println("Error ", err.Error())
} else {
// retrieve data directly
user := config.Get("postgres.user").(string)
password := config.Get("postgres.password").(string)
directUser := config.Get("postgres.user").(string)
directPassword := config.Get("postgres.password").(string)
fmt.Println("User is", directUser, " and password is", directPassword)
// or using an intermediate object
configTree := config.Get("postgres").(*toml.Tree)
user = configTree.Get("user").(string)
password = configTree.Get("password").(string)
user := configTree.Get("user").(string)
password := configTree.Get("password").(string)
fmt.Println("User is", user, " and password is", password)
// show where elements are in the file
Executable
+26
View File
@@ -0,0 +1,26 @@
#!/bin/bash
set -xe
# go-fuzz doesn't support modules yet, so ensure we do everything
# in the old style GOPATH way
export GO111MODULE="off"
# install go-fuzz
go get -u github.com/dvyukov/go-fuzz/go-fuzz github.com/dvyukov/go-fuzz/go-fuzz-build
# target name can only contain lower-case letters (a-z), digits (0-9) and a dash (-)
# to add another target, make sure to create it with `fuzzit create target`
# before using `fuzzit create job`
TARGET=toml-fuzzer
go-fuzz-build -libfuzzer -o ${TARGET}.a github.com/pelletier/go-toml
clang -fsanitize=fuzzer ${TARGET}.a -o ${TARGET}
# install fuzzit for talking to fuzzit.dev service
# or latest version:
# https://github.com/fuzzitdev/fuzzit/releases/latest/download/fuzzit_Linux_x86_64
wget -q -O fuzzit https://github.com/fuzzitdev/fuzzit/releases/download/v2.4.52/fuzzit_Linux_x86_64
chmod a+x fuzzit
# TODO: change kkowalczyk to go-toml and create toml-fuzzer target there
./fuzzit create job --type $TYPE go-toml/${TARGET} ${TARGET}
+1 -1
View File
@@ -5,5 +5,5 @@ go 1.12
require (
github.com/BurntSushi/toml v0.3.1
github.com/davecgh/go-spew v1.1.1
gopkg.in/yaml.v2 v2.2.2
gopkg.in/yaml.v2 v2.2.4
)
+4
View File
@@ -5,3 +5,7 @@ github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.3 h1:fvjTMHxHEw/mxHbtzPi3JCcKXQRAnQTBRo6YCJSVHKI=
gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4 h1:/eiJrUcujPVeJ3xlSWaiNi3uSVmDGBK1pDHUHAnao1I=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+1 -1
View File
@@ -24,7 +24,7 @@ func testResult(t *testing.T, key string, expected []string) {
func testError(t *testing.T, key string, expectedError string) {
res, err := parseKey(key)
if err == nil {
t.Fatalf("Expected error, but succesfully parsed key %s", res)
t.Fatalf("Expected error, but successfully parsed key %s", res)
}
if fmt.Sprintf("%s", err) != expectedError {
t.Fatalf("Expected error \"%s\", but got \"%s\".", expectedError, err)
+34 -6
View File
@@ -223,9 +223,12 @@ func (l *tomlLexer) lexRvalue() tomlLexStateFn {
}
possibleDate := l.peekString(35)
dateMatch := dateRegexp.FindString(possibleDate)
if dateMatch != "" {
l.fastForward(len(dateMatch))
dateSubmatches := dateRegexp.FindStringSubmatch(possibleDate)
if dateSubmatches != nil && dateSubmatches[0] != "" {
l.fastForward(len(dateSubmatches[0]))
if dateSubmatches[2] == "" { // no timezone information => local date
return l.lexLocalDate
}
return l.lexDate
}
@@ -247,13 +250,13 @@ func (l *tomlLexer) lexRvalue() tomlLexStateFn {
func (l *tomlLexer) lexLeftCurlyBrace() tomlLexStateFn {
l.next()
l.emit(tokenLeftCurlyBrace)
return l.lexRvalue
return l.lexVoid
}
func (l *tomlLexer) lexRightCurlyBrace() tomlLexStateFn {
l.next()
l.emit(tokenRightCurlyBrace)
return l.lexRvalue
return l.lexVoid
}
func (l *tomlLexer) lexDate() tomlLexStateFn {
@@ -261,6 +264,11 @@ func (l *tomlLexer) lexDate() tomlLexStateFn {
return l.lexRvalue
}
func (l *tomlLexer) lexLocalDate() tomlLexStateFn {
l.emit(tokenLocalDate)
return l.lexRvalue
}
func (l *tomlLexer) lexTrue() tomlLexStateFn {
l.fastForward(4)
l.emit(tokenTrue)
@@ -733,7 +741,27 @@ func (l *tomlLexer) run() {
}
func init() {
dateRegexp = regexp.MustCompile(`^\d{1,4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d{1,9})?(Z|[+-]\d{2}:\d{2})`)
// Regexp for all date/time formats supported by TOML.
// Group 1: nano precision
// Group 2: timezone
//
// /!\ also matches the empty string
//
// Example matches:
//1979-05-27T07:32:00Z
//1979-05-27T00:32:00-07:00
//1979-05-27T00:32:00.999999-07:00
//1979-05-27 07:32:00Z
//1979-05-27 00:32:00-07:00
//1979-05-27 00:32:00.999999-07:00
//1979-05-27T07:32:00
//1979-05-27T00:32:00.999999
//1979-05-27 07:32:00
//1979-05-27 00:32:00.999999
//1979-05-27
//07:32:00
//00:32:00.999999
dateRegexp = regexp.MustCompile(`^(?:\d{1,4}-\d{2}-\d{2})?(?:[T ]?\d{2}:\d{2}:\d{2}(\.\d{1,9})?(Z|[+-]\d{2}:\d{2})?)?`)
}
// Entry point
+79 -6
View File
@@ -290,14 +290,29 @@ func TestKeyEqualArrayBoolsWithComments(t *testing.T) {
}
func TestDateRegexp(t *testing.T) {
if dateRegexp.FindString("1979-05-27T07:32:00Z") == "" {
t.Error("basic lexing")
cases := map[string]string{
"basic": "1979-05-27T07:32:00Z",
"offset": "1979-05-27T00:32:00-07:00",
"nano precision": "1979-05-27T00:32:00.999999-07:00",
"basic-no-T": "1979-05-27 07:32:00Z",
"offset-no-T": "1979-05-27 00:32:00-07:00",
"nano precision-no-T": "1979-05-27 00:32:00.999999-07:00",
"no-tz": "1979-05-27T07:32:00",
"no-tz-nano": "1979-05-27T00:32:00.999999",
"no-tz-no-t": "1979-05-27 07:32:00",
"no-tz-no-t-nano": "1979-05-27 00:32:00.999999",
"date-no-tz": "1979-05-27",
"time-no-tz": "07:32:00",
"time-no-tz-nano": "00:32:00.999999",
}
if dateRegexp.FindString("1979-05-27T00:32:00-07:00") == "" {
t.Error("offset lexing")
for name, value := range cases {
if dateRegexp.FindString(value) == "" {
t.Error("failed date regexp test", name)
}
if dateRegexp.FindString("1979-05-27T00:32:00.999999-07:00") == "" {
t.Error("nano precision lexing")
}
if dateRegexp.FindString("1979-05-27 07:32:00Z") == "" {
t.Error("space delimiter lexing")
}
}
@@ -320,6 +335,12 @@ func TestKeyEqualDate(t *testing.T) {
{Position{1, 7}, tokenDate, "1979-05-27T00:32:00.999999-07:00"},
{Position{1, 39}, tokenEOF, ""},
})
testFlow(t, "foo = 1979-05-27 07:32:00Z", []token{
{Position{1, 1}, tokenKey, "foo"},
{Position{1, 5}, tokenEqual, "="},
{Position{1, 7}, tokenDate, "1979-05-27 07:32:00Z"},
{Position{1, 27}, tokenEOF, ""},
})
}
func TestFloatEndingWithDot(t *testing.T) {
@@ -726,6 +747,58 @@ func TestLexUnknownRvalue(t *testing.T) {
})
}
func TestLexInlineTableBareKey(t *testing.T) {
testFlow(t, `foo = { bar = "baz" }`, []token{
{Position{1, 1}, tokenKey, "foo"},
{Position{1, 5}, tokenEqual, "="},
{Position{1, 7}, tokenLeftCurlyBrace, "{"},
{Position{1, 9}, tokenKey, "bar"},
{Position{1, 13}, tokenEqual, "="},
{Position{1, 16}, tokenString, "baz"},
{Position{1, 21}, tokenRightCurlyBrace, "}"},
{Position{1, 22}, tokenEOF, ""},
})
}
func TestLexInlineTableBareKeyDash(t *testing.T) {
testFlow(t, `foo = { -bar = "baz" }`, []token{
{Position{1, 1}, tokenKey, "foo"},
{Position{1, 5}, tokenEqual, "="},
{Position{1, 7}, tokenLeftCurlyBrace, "{"},
{Position{1, 9}, tokenKey, "-bar"},
{Position{1, 14}, tokenEqual, "="},
{Position{1, 17}, tokenString, "baz"},
{Position{1, 22}, tokenRightCurlyBrace, "}"},
{Position{1, 23}, tokenEOF, ""},
})
}
func TestLexInlineTableBareKeyUnderscore(t *testing.T) {
testFlow(t, `foo = { _bar = "baz" }`, []token{
{Position{1, 1}, tokenKey, "foo"},
{Position{1, 5}, tokenEqual, "="},
{Position{1, 7}, tokenLeftCurlyBrace, "{"},
{Position{1, 9}, tokenKey, "_bar"},
{Position{1, 14}, tokenEqual, "="},
{Position{1, 17}, tokenString, "baz"},
{Position{1, 22}, tokenRightCurlyBrace, "}"},
{Position{1, 23}, tokenEOF, ""},
})
}
func TestLexInlineTableQuotedKey(t *testing.T) {
testFlow(t, `foo = { "bar" = "baz" }`, []token{
{Position{1, 1}, tokenKey, "foo"},
{Position{1, 5}, tokenEqual, "="},
{Position{1, 7}, tokenLeftCurlyBrace, "{"},
{Position{1, 9}, tokenKey, "\"bar\""},
{Position{1, 15}, tokenEqual, "="},
{Position{1, 18}, tokenString, "baz"},
{Position{1, 23}, tokenRightCurlyBrace, "}"},
{Position{1, 24}, tokenEOF, ""},
})
}
func BenchmarkLexer(b *testing.B) {
sample := `title = "Hugo: A Fast and Flexible Website Generator"
baseurl = "http://gohugo.io/"
+281
View File
@@ -0,0 +1,281 @@
// Implementation of TOML's local date/time.
// Copied over from https://github.com/googleapis/google-cloud-go/blob/master/civil/civil.go
// to avoid pulling all the Google dependencies.
//
// Copyright 2016 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package civil implements types for civil time, a time-zone-independent
// representation of time that follows the rules of the proleptic
// Gregorian calendar with exactly 24-hour days, 60-minute hours, and 60-second
// minutes.
//
// Because they lack location information, these types do not represent unique
// moments or intervals of time. Use time.Time for that purpose.
package toml
import (
"fmt"
"time"
)
// A LocalDate represents a date (year, month, day).
//
// This type does not include location information, and therefore does not
// describe a unique 24-hour timespan.
type LocalDate struct {
Year int // Year (e.g., 2014).
Month time.Month // Month of the year (January = 1, ...).
Day int // Day of the month, starting at 1.
}
// LocalDateOf returns the LocalDate in which a time occurs in that time's location.
func LocalDateOf(t time.Time) LocalDate {
var d LocalDate
d.Year, d.Month, d.Day = t.Date()
return d
}
// ParseLocalDate parses a string in RFC3339 full-date format and returns the date value it represents.
func ParseLocalDate(s string) (LocalDate, error) {
t, err := time.Parse("2006-01-02", s)
if err != nil {
return LocalDate{}, err
}
return LocalDateOf(t), nil
}
// String returns the date in RFC3339 full-date format.
func (d LocalDate) String() string {
return fmt.Sprintf("%04d-%02d-%02d", d.Year, d.Month, d.Day)
}
// IsValid reports whether the date is valid.
func (d LocalDate) IsValid() bool {
return LocalDateOf(d.In(time.UTC)) == d
}
// In returns the time corresponding to time 00:00:00 of the date in the location.
//
// In is always consistent with time.LocalDate, even when time.LocalDate returns a time
// on a different day. For example, if loc is America/Indiana/Vincennes, then both
// time.LocalDate(1955, time.May, 1, 0, 0, 0, 0, loc)
// and
// civil.LocalDate{Year: 1955, Month: time.May, Day: 1}.In(loc)
// return 23:00:00 on April 30, 1955.
//
// In panics if loc is nil.
func (d LocalDate) In(loc *time.Location) time.Time {
return time.Date(d.Year, d.Month, d.Day, 0, 0, 0, 0, loc)
}
// AddDays returns the date that is n days in the future.
// n can also be negative to go into the past.
func (d LocalDate) AddDays(n int) LocalDate {
return LocalDateOf(d.In(time.UTC).AddDate(0, 0, n))
}
// DaysSince returns the signed number of days between the date and s, not including the end day.
// This is the inverse operation to AddDays.
func (d LocalDate) DaysSince(s LocalDate) (days int) {
// We convert to Unix time so we do not have to worry about leap seconds:
// Unix time increases by exactly 86400 seconds per day.
deltaUnix := d.In(time.UTC).Unix() - s.In(time.UTC).Unix()
return int(deltaUnix / 86400)
}
// Before reports whether d1 occurs before d2.
func (d1 LocalDate) Before(d2 LocalDate) bool {
if d1.Year != d2.Year {
return d1.Year < d2.Year
}
if d1.Month != d2.Month {
return d1.Month < d2.Month
}
return d1.Day < d2.Day
}
// After reports whether d1 occurs after d2.
func (d1 LocalDate) After(d2 LocalDate) bool {
return d2.Before(d1)
}
// MarshalText implements the encoding.TextMarshaler interface.
// The output is the result of d.String().
func (d LocalDate) MarshalText() ([]byte, error) {
return []byte(d.String()), nil
}
// UnmarshalText implements the encoding.TextUnmarshaler interface.
// The date is expected to be a string in a format accepted by ParseLocalDate.
func (d *LocalDate) UnmarshalText(data []byte) error {
var err error
*d, err = ParseLocalDate(string(data))
return err
}
// A LocalTime represents a time with nanosecond precision.
//
// This type does not include location information, and therefore does not
// describe a unique moment in time.
//
// This type exists to represent the TIME type in storage-based APIs like BigQuery.
// Most operations on Times are unlikely to be meaningful. Prefer the LocalDateTime type.
type LocalTime struct {
Hour int // The hour of the day in 24-hour format; range [0-23]
Minute int // The minute of the hour; range [0-59]
Second int // The second of the minute; range [0-59]
Nanosecond int // The nanosecond of the second; range [0-999999999]
}
// LocalTimeOf returns the LocalTime representing the time of day in which a time occurs
// in that time's location. It ignores the date.
func LocalTimeOf(t time.Time) LocalTime {
var tm LocalTime
tm.Hour, tm.Minute, tm.Second = t.Clock()
tm.Nanosecond = t.Nanosecond()
return tm
}
// ParseLocalTime parses a string and returns the time value it represents.
// ParseLocalTime accepts an extended form of the RFC3339 partial-time format. After
// the HH:MM:SS part of the string, an optional fractional part may appear,
// consisting of a decimal point followed by one to nine decimal digits.
// (RFC3339 admits only one digit after the decimal point).
func ParseLocalTime(s string) (LocalTime, error) {
t, err := time.Parse("15:04:05.999999999", s)
if err != nil {
return LocalTime{}, err
}
return LocalTimeOf(t), nil
}
// String returns the date in the format described in ParseLocalTime. If Nanoseconds
// is zero, no fractional part will be generated. Otherwise, the result will
// end with a fractional part consisting of a decimal point and nine digits.
func (t LocalTime) String() string {
s := fmt.Sprintf("%02d:%02d:%02d", t.Hour, t.Minute, t.Second)
if t.Nanosecond == 0 {
return s
}
return s + fmt.Sprintf(".%09d", t.Nanosecond)
}
// IsValid reports whether the time is valid.
func (t LocalTime) IsValid() bool {
// Construct a non-zero time.
tm := time.Date(2, 2, 2, t.Hour, t.Minute, t.Second, t.Nanosecond, time.UTC)
return LocalTimeOf(tm) == t
}
// MarshalText implements the encoding.TextMarshaler interface.
// The output is the result of t.String().
func (t LocalTime) MarshalText() ([]byte, error) {
return []byte(t.String()), nil
}
// UnmarshalText implements the encoding.TextUnmarshaler interface.
// The time is expected to be a string in a format accepted by ParseLocalTime.
func (t *LocalTime) UnmarshalText(data []byte) error {
var err error
*t, err = ParseLocalTime(string(data))
return err
}
// A LocalDateTime represents a date and time.
//
// This type does not include location information, and therefore does not
// describe a unique moment in time.
type LocalDateTime struct {
Date LocalDate
Time LocalTime
}
// Note: We deliberately do not embed LocalDate into LocalDateTime, to avoid promoting AddDays and Sub.
// LocalDateTimeOf returns the LocalDateTime in which a time occurs in that time's location.
func LocalDateTimeOf(t time.Time) LocalDateTime {
return LocalDateTime{
Date: LocalDateOf(t),
Time: LocalTimeOf(t),
}
}
// ParseLocalDateTime parses a string and returns the LocalDateTime it represents.
// ParseLocalDateTime accepts a variant of the RFC3339 date-time format that omits
// the time offset but includes an optional fractional time, as described in
// ParseLocalTime. Informally, the accepted format is
// YYYY-MM-DDTHH:MM:SS[.FFFFFFFFF]
// where the 'T' may be a lower-case 't'.
func ParseLocalDateTime(s string) (LocalDateTime, error) {
t, err := time.Parse("2006-01-02T15:04:05.999999999", s)
if err != nil {
t, err = time.Parse("2006-01-02t15:04:05.999999999", s)
if err != nil {
return LocalDateTime{}, err
}
}
return LocalDateTimeOf(t), nil
}
// String returns the date in the format described in ParseLocalDate.
func (dt LocalDateTime) String() string {
return dt.Date.String() + "T" + dt.Time.String()
}
// IsValid reports whether the datetime is valid.
func (dt LocalDateTime) IsValid() bool {
return dt.Date.IsValid() && dt.Time.IsValid()
}
// In returns the time corresponding to the LocalDateTime in the given location.
//
// If the time is missing or ambigous at the location, In returns the same
// result as time.LocalDate. For example, if loc is America/Indiana/Vincennes, then
// both
// time.LocalDate(1955, time.May, 1, 0, 30, 0, 0, loc)
// and
// civil.LocalDateTime{
// civil.LocalDate{Year: 1955, Month: time.May, Day: 1}},
// civil.LocalTime{Minute: 30}}.In(loc)
// return 23:30:00 on April 30, 1955.
//
// In panics if loc is nil.
func (dt LocalDateTime) In(loc *time.Location) time.Time {
return time.Date(dt.Date.Year, dt.Date.Month, dt.Date.Day, dt.Time.Hour, dt.Time.Minute, dt.Time.Second, dt.Time.Nanosecond, loc)
}
// Before reports whether dt1 occurs before dt2.
func (dt1 LocalDateTime) Before(dt2 LocalDateTime) bool {
return dt1.In(time.UTC).Before(dt2.In(time.UTC))
}
// After reports whether dt1 occurs after dt2.
func (dt1 LocalDateTime) After(dt2 LocalDateTime) bool {
return dt2.Before(dt1)
}
// MarshalText implements the encoding.TextMarshaler interface.
// The output is the result of dt.String().
func (dt LocalDateTime) MarshalText() ([]byte, error) {
return []byte(dt.String()), nil
}
// UnmarshalText implements the encoding.TextUnmarshaler interface.
// The datetime is expected to be a string in a format accepted by ParseLocalDateTime
func (dt *LocalDateTime) UnmarshalText(data []byte) error {
var err error
*dt, err = ParseLocalDateTime(string(data))
return err
}
+446
View File
@@ -0,0 +1,446 @@
// Copyright 2016 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package toml
import (
"encoding/json"
"reflect"
"testing"
"time"
)
func cmpEqual(x, y interface{}) bool {
return reflect.DeepEqual(x, y)
}
func TestDates(t *testing.T) {
for _, test := range []struct {
date LocalDate
loc *time.Location
wantStr string
wantTime time.Time
}{
{
date: LocalDate{2014, 7, 29},
loc: time.Local,
wantStr: "2014-07-29",
wantTime: time.Date(2014, time.July, 29, 0, 0, 0, 0, time.Local),
},
{
date: LocalDateOf(time.Date(2014, 8, 20, 15, 8, 43, 1, time.Local)),
loc: time.UTC,
wantStr: "2014-08-20",
wantTime: time.Date(2014, 8, 20, 0, 0, 0, 0, time.UTC),
},
{
date: LocalDateOf(time.Date(999, time.January, 26, 0, 0, 0, 0, time.Local)),
loc: time.UTC,
wantStr: "0999-01-26",
wantTime: time.Date(999, 1, 26, 0, 0, 0, 0, time.UTC),
},
} {
if got := test.date.String(); got != test.wantStr {
t.Errorf("%#v.String() = %q, want %q", test.date, got, test.wantStr)
}
if got := test.date.In(test.loc); !got.Equal(test.wantTime) {
t.Errorf("%#v.In(%v) = %v, want %v", test.date, test.loc, got, test.wantTime)
}
}
}
func TestDateIsValid(t *testing.T) {
for _, test := range []struct {
date LocalDate
want bool
}{
{LocalDate{2014, 7, 29}, true},
{LocalDate{2000, 2, 29}, true},
{LocalDate{10000, 12, 31}, true},
{LocalDate{1, 1, 1}, true},
{LocalDate{0, 1, 1}, true}, // year zero is OK
{LocalDate{-1, 1, 1}, true}, // negative year is OK
{LocalDate{1, 0, 1}, false},
{LocalDate{1, 1, 0}, false},
{LocalDate{2016, 1, 32}, false},
{LocalDate{2016, 13, 1}, false},
{LocalDate{1, -1, 1}, false},
{LocalDate{1, 1, -1}, false},
} {
got := test.date.IsValid()
if got != test.want {
t.Errorf("%#v: got %t, want %t", test.date, got, test.want)
}
}
}
func TestParseDate(t *testing.T) {
for _, test := range []struct {
str string
want LocalDate // if empty, expect an error
}{
{"2016-01-02", LocalDate{2016, 1, 2}},
{"2016-12-31", LocalDate{2016, 12, 31}},
{"0003-02-04", LocalDate{3, 2, 4}},
{"999-01-26", LocalDate{}},
{"", LocalDate{}},
{"2016-01-02x", LocalDate{}},
} {
got, err := ParseLocalDate(test.str)
if got != test.want {
t.Errorf("ParseLocalDate(%q) = %+v, want %+v", test.str, got, test.want)
}
if err != nil && test.want != (LocalDate{}) {
t.Errorf("Unexpected error %v from ParseLocalDate(%q)", err, test.str)
}
}
}
func TestDateArithmetic(t *testing.T) {
for _, test := range []struct {
desc string
start LocalDate
end LocalDate
days int
}{
{
desc: "zero days noop",
start: LocalDate{2014, 5, 9},
end: LocalDate{2014, 5, 9},
days: 0,
},
{
desc: "crossing a year boundary",
start: LocalDate{2014, 12, 31},
end: LocalDate{2015, 1, 1},
days: 1,
},
{
desc: "negative number of days",
start: LocalDate{2015, 1, 1},
end: LocalDate{2014, 12, 31},
days: -1,
},
{
desc: "full leap year",
start: LocalDate{2004, 1, 1},
end: LocalDate{2005, 1, 1},
days: 366,
},
{
desc: "full non-leap year",
start: LocalDate{2001, 1, 1},
end: LocalDate{2002, 1, 1},
days: 365,
},
{
desc: "crossing a leap second",
start: LocalDate{1972, 6, 30},
end: LocalDate{1972, 7, 1},
days: 1,
},
{
desc: "dates before the unix epoch",
start: LocalDate{101, 1, 1},
end: LocalDate{102, 1, 1},
days: 365,
},
} {
if got := test.start.AddDays(test.days); got != test.end {
t.Errorf("[%s] %#v.AddDays(%v) = %#v, want %#v", test.desc, test.start, test.days, got, test.end)
}
if got := test.end.DaysSince(test.start); got != test.days {
t.Errorf("[%s] %#v.Sub(%#v) = %v, want %v", test.desc, test.end, test.start, got, test.days)
}
}
}
func TestDateBefore(t *testing.T) {
for _, test := range []struct {
d1, d2 LocalDate
want bool
}{
{LocalDate{2016, 12, 31}, LocalDate{2017, 1, 1}, true},
{LocalDate{2016, 1, 1}, LocalDate{2016, 1, 1}, false},
{LocalDate{2016, 12, 30}, LocalDate{2016, 12, 31}, true},
{LocalDate{2016, 1, 30}, LocalDate{2016, 12, 31}, true},
} {
if got := test.d1.Before(test.d2); got != test.want {
t.Errorf("%v.Before(%v): got %t, want %t", test.d1, test.d2, got, test.want)
}
}
}
func TestDateAfter(t *testing.T) {
for _, test := range []struct {
d1, d2 LocalDate
want bool
}{
{LocalDate{2016, 12, 31}, LocalDate{2017, 1, 1}, false},
{LocalDate{2016, 1, 1}, LocalDate{2016, 1, 1}, false},
{LocalDate{2016, 12, 30}, LocalDate{2016, 12, 31}, false},
} {
if got := test.d1.After(test.d2); got != test.want {
t.Errorf("%v.After(%v): got %t, want %t", test.d1, test.d2, got, test.want)
}
}
}
func TestTimeToString(t *testing.T) {
for _, test := range []struct {
str string
time LocalTime
roundTrip bool // ParseLocalTime(str).String() == str?
}{
{"13:26:33", LocalTime{13, 26, 33, 0}, true},
{"01:02:03.000023456", LocalTime{1, 2, 3, 23456}, true},
{"00:00:00.000000001", LocalTime{0, 0, 0, 1}, true},
{"13:26:03.1", LocalTime{13, 26, 3, 100000000}, false},
{"13:26:33.0000003", LocalTime{13, 26, 33, 300}, false},
} {
gotTime, err := ParseLocalTime(test.str)
if err != nil {
t.Errorf("ParseLocalTime(%q): got error: %v", test.str, err)
continue
}
if gotTime != test.time {
t.Errorf("ParseLocalTime(%q) = %+v, want %+v", test.str, gotTime, test.time)
}
if test.roundTrip {
gotStr := test.time.String()
if gotStr != test.str {
t.Errorf("%#v.String() = %q, want %q", test.time, gotStr, test.str)
}
}
}
}
func TestTimeOf(t *testing.T) {
for _, test := range []struct {
time time.Time
want LocalTime
}{
{time.Date(2014, 8, 20, 15, 8, 43, 1, time.Local), LocalTime{15, 8, 43, 1}},
{time.Date(1, 1, 1, 0, 0, 0, 0, time.UTC), LocalTime{0, 0, 0, 0}},
} {
if got := LocalTimeOf(test.time); got != test.want {
t.Errorf("LocalTimeOf(%v) = %+v, want %+v", test.time, got, test.want)
}
}
}
func TestTimeIsValid(t *testing.T) {
for _, test := range []struct {
time LocalTime
want bool
}{
{LocalTime{0, 0, 0, 0}, true},
{LocalTime{23, 0, 0, 0}, true},
{LocalTime{23, 59, 59, 999999999}, true},
{LocalTime{24, 59, 59, 999999999}, false},
{LocalTime{23, 60, 59, 999999999}, false},
{LocalTime{23, 59, 60, 999999999}, false},
{LocalTime{23, 59, 59, 1000000000}, false},
{LocalTime{-1, 0, 0, 0}, false},
{LocalTime{0, -1, 0, 0}, false},
{LocalTime{0, 0, -1, 0}, false},
{LocalTime{0, 0, 0, -1}, false},
} {
got := test.time.IsValid()
if got != test.want {
t.Errorf("%#v: got %t, want %t", test.time, got, test.want)
}
}
}
func TestDateTimeToString(t *testing.T) {
for _, test := range []struct {
str string
dateTime LocalDateTime
roundTrip bool // ParseLocalDateTime(str).String() == str?
}{
{"2016-03-22T13:26:33", LocalDateTime{LocalDate{2016, 03, 22}, LocalTime{13, 26, 33, 0}}, true},
{"2016-03-22T13:26:33.000000600", LocalDateTime{LocalDate{2016, 03, 22}, LocalTime{13, 26, 33, 600}}, true},
{"2016-03-22t13:26:33", LocalDateTime{LocalDate{2016, 03, 22}, LocalTime{13, 26, 33, 0}}, false},
} {
gotDateTime, err := ParseLocalDateTime(test.str)
if err != nil {
t.Errorf("ParseLocalDateTime(%q): got error: %v", test.str, err)
continue
}
if gotDateTime != test.dateTime {
t.Errorf("ParseLocalDateTime(%q) = %+v, want %+v", test.str, gotDateTime, test.dateTime)
}
if test.roundTrip {
gotStr := test.dateTime.String()
if gotStr != test.str {
t.Errorf("%#v.String() = %q, want %q", test.dateTime, gotStr, test.str)
}
}
}
}
func TestParseDateTimeErrors(t *testing.T) {
for _, str := range []string{
"",
"2016-03-22", // just a date
"13:26:33", // just a time
"2016-03-22 13:26:33", // wrong separating character
"2016-03-22T13:26:33x", // extra at end
} {
if _, err := ParseLocalDateTime(str); err == nil {
t.Errorf("ParseLocalDateTime(%q) succeeded, want error", str)
}
}
}
func TestDateTimeOf(t *testing.T) {
for _, test := range []struct {
time time.Time
want LocalDateTime
}{
{time.Date(2014, 8, 20, 15, 8, 43, 1, time.Local),
LocalDateTime{LocalDate{2014, 8, 20}, LocalTime{15, 8, 43, 1}}},
{time.Date(1, 1, 1, 0, 0, 0, 0, time.UTC),
LocalDateTime{LocalDate{1, 1, 1}, LocalTime{0, 0, 0, 0}}},
} {
if got := LocalDateTimeOf(test.time); got != test.want {
t.Errorf("LocalDateTimeOf(%v) = %+v, want %+v", test.time, got, test.want)
}
}
}
func TestDateTimeIsValid(t *testing.T) {
// No need to be exhaustive here; it's just LocalDate.IsValid && LocalTime.IsValid.
for _, test := range []struct {
dt LocalDateTime
want bool
}{
{LocalDateTime{LocalDate{2016, 3, 20}, LocalTime{0, 0, 0, 0}}, true},
{LocalDateTime{LocalDate{2016, -3, 20}, LocalTime{0, 0, 0, 0}}, false},
{LocalDateTime{LocalDate{2016, 3, 20}, LocalTime{24, 0, 0, 0}}, false},
} {
got := test.dt.IsValid()
if got != test.want {
t.Errorf("%#v: got %t, want %t", test.dt, got, test.want)
}
}
}
func TestDateTimeIn(t *testing.T) {
dt := LocalDateTime{LocalDate{2016, 1, 2}, LocalTime{3, 4, 5, 6}}
got := dt.In(time.UTC)
want := time.Date(2016, 1, 2, 3, 4, 5, 6, time.UTC)
if !got.Equal(want) {
t.Errorf("got %v, want %v", got, want)
}
}
func TestDateTimeBefore(t *testing.T) {
d1 := LocalDate{2016, 12, 31}
d2 := LocalDate{2017, 1, 1}
t1 := LocalTime{5, 6, 7, 8}
t2 := LocalTime{5, 6, 7, 9}
for _, test := range []struct {
dt1, dt2 LocalDateTime
want bool
}{
{LocalDateTime{d1, t1}, LocalDateTime{d2, t1}, true},
{LocalDateTime{d1, t1}, LocalDateTime{d1, t2}, true},
{LocalDateTime{d2, t1}, LocalDateTime{d1, t1}, false},
{LocalDateTime{d2, t1}, LocalDateTime{d2, t1}, false},
} {
if got := test.dt1.Before(test.dt2); got != test.want {
t.Errorf("%v.Before(%v): got %t, want %t", test.dt1, test.dt2, got, test.want)
}
}
}
func TestDateTimeAfter(t *testing.T) {
d1 := LocalDate{2016, 12, 31}
d2 := LocalDate{2017, 1, 1}
t1 := LocalTime{5, 6, 7, 8}
t2 := LocalTime{5, 6, 7, 9}
for _, test := range []struct {
dt1, dt2 LocalDateTime
want bool
}{
{LocalDateTime{d1, t1}, LocalDateTime{d2, t1}, false},
{LocalDateTime{d1, t1}, LocalDateTime{d1, t2}, false},
{LocalDateTime{d2, t1}, LocalDateTime{d1, t1}, true},
{LocalDateTime{d2, t1}, LocalDateTime{d2, t1}, false},
} {
if got := test.dt1.After(test.dt2); got != test.want {
t.Errorf("%v.After(%v): got %t, want %t", test.dt1, test.dt2, got, test.want)
}
}
}
func TestMarshalJSON(t *testing.T) {
for _, test := range []struct {
value interface{}
want string
}{
{LocalDate{1987, 4, 15}, `"1987-04-15"`},
{LocalTime{18, 54, 2, 0}, `"18:54:02"`},
{LocalDateTime{LocalDate{1987, 4, 15}, LocalTime{18, 54, 2, 0}}, `"1987-04-15T18:54:02"`},
} {
bgot, err := json.Marshal(test.value)
if err != nil {
t.Fatal(err)
}
if got := string(bgot); got != test.want {
t.Errorf("%#v: got %s, want %s", test.value, got, test.want)
}
}
}
func TestUnmarshalJSON(t *testing.T) {
var d LocalDate
var tm LocalTime
var dt LocalDateTime
for _, test := range []struct {
data string
ptr interface{}
want interface{}
}{
{`"1987-04-15"`, &d, &LocalDate{1987, 4, 15}},
{`"1987-04-\u0031\u0035"`, &d, &LocalDate{1987, 4, 15}},
{`"18:54:02"`, &tm, &LocalTime{18, 54, 2, 0}},
{`"1987-04-15T18:54:02"`, &dt, &LocalDateTime{LocalDate{1987, 4, 15}, LocalTime{18, 54, 2, 0}}},
} {
if err := json.Unmarshal([]byte(test.data), test.ptr); err != nil {
t.Fatalf("%s: %v", test.data, err)
}
if !cmpEqual(test.ptr, test.want) {
t.Errorf("%s: got %#v, want %#v", test.data, test.ptr, test.want)
}
}
for _, bad := range []string{"", `""`, `"bad"`, `"1987-04-15x"`,
`19870415`, // a JSON number
`11987-04-15x`, // not a JSON string
} {
if json.Unmarshal([]byte(bad), &d) == nil {
t.Errorf("%q, LocalDate: got nil, want error", bad)
}
if json.Unmarshal([]byte(bad), &tm) == nil {
t.Errorf("%q, LocalTime: got nil, want error", bad)
}
if json.Unmarshal([]byte(bad), &dt) == nil {
t.Errorf("%q, LocalDateTime: got nil, want error", bad)
}
}
}
+138 -45
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"io"
"reflect"
"sort"
"strconv"
"strings"
"time"
@@ -67,6 +68,9 @@ const (
var timeType = reflect.TypeOf(time.Time{})
var marshalerType = reflect.TypeOf(new(Marshaler)).Elem()
var localDateType = reflect.TypeOf(LocalDate{})
var localTimeType = reflect.TypeOf(LocalTime{})
var localDateTimeType = reflect.TypeOf(LocalDateTime{})
// Check if the given marshal type maps to a Tree primitive
func isPrimitive(mtype reflect.Type) bool {
@@ -84,29 +88,31 @@ func isPrimitive(mtype reflect.Type) bool {
case reflect.String:
return true
case reflect.Struct:
return mtype == timeType || isCustomMarshaler(mtype)
return mtype == timeType || mtype == localDateType || mtype == localDateTimeType || mtype == localTimeType || isCustomMarshaler(mtype)
default:
return false
}
}
// Check if the given marshal type maps to a Tree slice
func isTreeSlice(mtype reflect.Type) bool {
switch mtype.Kind() {
case reflect.Slice:
return !isOtherSlice(mtype)
default:
return false
}
}
// Check if the given marshal type maps to a non-Tree slice
func isOtherSlice(mtype reflect.Type) bool {
// Check if the given marshal type maps to a Tree slice or array
func isTreeSequence(mtype reflect.Type) bool {
switch mtype.Kind() {
case reflect.Ptr:
return isOtherSlice(mtype.Elem())
case reflect.Slice:
return isPrimitive(mtype.Elem()) || isOtherSlice(mtype.Elem())
return isTreeSequence(mtype.Elem())
case reflect.Slice, reflect.Array:
return isTree(mtype.Elem())
default:
return false
}
}
// Check if the given marshal type maps to a non-Tree slice or array
func isOtherSequence(mtype reflect.Type) bool {
switch mtype.Kind() {
case reflect.Ptr:
return isOtherSequence(mtype.Elem())
case reflect.Slice, reflect.Array:
return !isTreeSequence(mtype)
default:
return false
}
@@ -115,6 +121,8 @@ func isOtherSlice(mtype reflect.Type) bool {
// Check if the given marshal type maps to a Tree
func isTree(mtype reflect.Type) bool {
switch mtype.Kind() {
case reflect.Ptr:
return isTree(mtype.Elem())
case reflect.Map:
return true
case reflect.Struct:
@@ -169,7 +177,7 @@ Tree primitive types and corresponding marshal types:
float64 float32, float64, pointers to same
string string, pointers to same
bool bool, pointers to same
time.Time time.Time{}, pointers to same
time.LocalTime time.LocalTime{}, pointers to same
For additional flexibility, use the Encoder API.
*/
@@ -329,7 +337,26 @@ func (e *Encoder) valueToTree(mtype reflect.Type, mval reflect.Value) (*Tree, er
}
}
case reflect.Map:
for _, key := range mval.MapKeys() {
keys := mval.MapKeys()
if e.order == OrderPreserve && len(keys) > 0 {
// Sorting []reflect.Value is not straight forward.
//
// OrderPreserve will support deterministic results when string is used
// as the key to maps.
typ := keys[0].Type()
kind := keys[0].Kind()
if kind == reflect.String {
ikeys := make([]string, len(keys))
for i := range keys {
ikeys[i] = keys[i].Interface().(string)
}
sort.Strings(ikeys)
for i := range ikeys {
keys[i] = reflect.ValueOf(ikeys[i]).Convert(typ)
}
}
}
for _, key := range keys {
mvalf := mval.MapIndex(key)
val, err := e.valueToToml(mtype.Elem(), mvalf)
if err != nil {
@@ -386,9 +413,9 @@ func (e *Encoder) valueToToml(mtype reflect.Type, mval reflect.Value) (interface
return callCustomMarshaler(mval)
case isTree(mtype):
return e.valueToTree(mtype, mval)
case isTreeSlice(mtype):
case isTreeSequence(mtype):
return e.valueToTreeSlice(mtype, mval)
case isOtherSlice(mtype):
case isOtherSequence(mtype):
return e.valueToOtherSlice(mtype, mval)
default:
switch mtype.Kind() {
@@ -406,7 +433,7 @@ func (e *Encoder) valueToToml(mtype reflect.Type, mval reflect.Value) (interface
case reflect.String:
return mval.String(), nil
case reflect.Struct:
return mval.Interface().(time.Time), nil
return mval.Interface(), nil
default:
return nil, fmt.Errorf("Marshal can't handle %v(%v)", mtype, mtype.Kind())
}
@@ -425,8 +452,11 @@ func (t *Tree) Unmarshal(v interface{}) error {
// See Marshal() documentation for types mapping table.
func (t *Tree) Marshal() ([]byte, error) {
var buf bytes.Buffer
err := NewEncoder(&buf).Encode(t)
return buf.Bytes(), err
_, err := t.WriteTo(&buf)
if err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// Unmarshal parses the TOML-encoded data and stores the result in the value
@@ -494,11 +524,21 @@ func (d *Decoder) SetTagName(v string) *Decoder {
func (d *Decoder) unmarshal(v interface{}) error {
mtype := reflect.TypeOf(v)
if mtype.Kind() != reflect.Ptr || mtype.Elem().Kind() != reflect.Struct {
return errors.New("Only a pointer to struct can be unmarshaled from TOML")
if mtype.Kind() != reflect.Ptr {
return errors.New("only a pointer to struct or map can be unmarshaled from TOML")
}
sval, err := d.valueFromTree(mtype.Elem(), d.tval)
elem := mtype.Elem()
switch elem.Kind() {
case reflect.Struct, reflect.Map:
default:
return errors.New("only a pointer to struct or map can be unmarshaled from TOML")
}
vv := reflect.ValueOf(v).Elem()
sval, err := d.valueFromTree(elem, d.tval, &vv)
if err != nil {
return err
}
@@ -506,15 +546,21 @@ func (d *Decoder) unmarshal(v interface{}) error {
return nil
}
// Convert toml tree to marshal struct or map, using marshal type
func (d *Decoder) valueFromTree(mtype reflect.Type, tval *Tree) (reflect.Value, error) {
// Convert toml tree to marshal struct or map, using marshal type. When mval1
// is non-nil, merge fields into the given value instead of allocating a new one.
func (d *Decoder) valueFromTree(mtype reflect.Type, tval *Tree, mval1 *reflect.Value) (reflect.Value, error) {
if mtype.Kind() == reflect.Ptr {
return d.unwrapPointer(mtype, tval)
return d.unwrapPointer(mtype, tval, mval1)
}
var mval reflect.Value
switch mtype.Kind() {
case reflect.Struct:
if mval1 != nil {
mval = *mval1
} else {
mval = reflect.New(mtype).Elem()
}
for i := 0; i < mtype.NumField(); i++ {
mtypef := mtype.Field(i)
an := annotation{tag: d.tagName}
@@ -535,7 +581,8 @@ func (d *Decoder) valueFromTree(mtype reflect.Type, tval *Tree) (reflect.Value,
continue
}
val := tval.Get(key)
mvalf, err := d.valueFromToml(mtypef.Type, val)
fval := mval.Field(i)
mvalf, err := d.valueFromToml(mtypef.Type, val, &fval)
if err != nil {
return mval, formatError(err, tval.GetPosition(key))
}
@@ -546,8 +593,8 @@ func (d *Decoder) valueFromTree(mtype reflect.Type, tval *Tree) (reflect.Value,
if !found && opts.defaultValue != "" {
mvalf := mval.Field(i)
var val interface{} = nil
var err error = nil
var val interface{}
var err error
switch mvalf.Kind() {
case reflect.Bool:
val, err = strconv.ParseBool(opts.defaultValue)
@@ -576,6 +623,15 @@ func (d *Decoder) valueFromTree(mtype reflect.Type, tval *Tree) (reflect.Value,
}
mval.Field(i).Set(reflect.ValueOf(val))
}
// save the old behavior above and try to check anonymous structs
if !found && opts.defaultValue == "" && mtypef.Anonymous && mtypef.Type.Kind() == reflect.Struct {
v, err := d.valueFromTree(mtypef.Type, tval, nil)
if err != nil {
return v, err
}
mval.Field(i).Set(v)
}
}
}
case reflect.Map:
@@ -583,11 +639,11 @@ func (d *Decoder) valueFromTree(mtype reflect.Type, tval *Tree) (reflect.Value,
for _, key := range tval.Keys() {
// TODO: path splits key
val := tval.GetPath([]string{key})
mvalf, err := d.valueFromToml(mtype.Elem(), val)
mvalf, err := d.valueFromToml(mtype.Elem(), val, nil)
if err != nil {
return mval, formatError(err, tval.GetPosition(key))
}
mval.SetMapIndex(reflect.ValueOf(key), mvalf)
mval.SetMapIndex(reflect.ValueOf(key).Convert(mtype.Key()), mvalf)
}
}
return mval, nil
@@ -597,7 +653,7 @@ func (d *Decoder) valueFromTree(mtype reflect.Type, tval *Tree) (reflect.Value,
func (d *Decoder) valueFromTreeSlice(mtype reflect.Type, tval []*Tree) (reflect.Value, error) {
mval := reflect.MakeSlice(mtype, len(tval), len(tval))
for i := 0; i < len(tval); i++ {
val, err := d.valueFromTree(mtype.Elem(), tval[i])
val, err := d.valueFromTree(mtype.Elem(), tval[i], nil)
if err != nil {
return mval, err
}
@@ -610,7 +666,7 @@ func (d *Decoder) valueFromTreeSlice(mtype reflect.Type, tval []*Tree) (reflect.
func (d *Decoder) valueFromOtherSlice(mtype reflect.Type, tval []interface{}) (reflect.Value, error) {
mval := reflect.MakeSlice(mtype, len(tval), len(tval))
for i := 0; i < len(tval); i++ {
val, err := d.valueFromToml(mtype.Elem(), tval[i])
val, err := d.valueFromToml(mtype.Elem(), tval[i], nil)
if err != nil {
return mval, err
}
@@ -619,25 +675,31 @@ func (d *Decoder) valueFromOtherSlice(mtype reflect.Type, tval []interface{}) (r
return mval, nil
}
// Convert toml value to marshal value, using marshal type
func (d *Decoder) valueFromToml(mtype reflect.Type, tval interface{}) (reflect.Value, error) {
// Convert toml value to marshal value, using marshal type. When mval1 is non-nil
// and the given type is a struct value, merge fields into it.
func (d *Decoder) valueFromToml(mtype reflect.Type, tval interface{}, mval1 *reflect.Value) (reflect.Value, error) {
if mtype.Kind() == reflect.Ptr {
return d.unwrapPointer(mtype, tval)
return d.unwrapPointer(mtype, tval, mval1)
}
switch t := tval.(type) {
case *Tree:
var mval11 *reflect.Value
if mtype.Kind() == reflect.Struct {
mval11 = mval1
}
if isTree(mtype) {
return d.valueFromTree(mtype, t)
return d.valueFromTree(mtype, t, mval11)
}
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to a tree", tval, tval)
case []*Tree:
if isTreeSlice(mtype) {
if isTreeSequence(mtype) {
return d.valueFromTreeSlice(mtype, t)
}
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to trees", tval, tval)
case []interface{}:
if isOtherSlice(mtype) {
if isOtherSequence(mtype) {
return d.valueFromOtherSlice(mtype, t)
}
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to a slice", tval, tval)
@@ -645,7 +707,31 @@ func (d *Decoder) valueFromToml(mtype reflect.Type, tval interface{}) (reflect.V
switch mtype.Kind() {
case reflect.Bool, reflect.Struct:
val := reflect.ValueOf(tval)
// if this passes for when mtype is reflect.Struct, tval is a time.Time
switch val.Type() {
case localDateType:
localDate := val.Interface().(LocalDate)
switch mtype {
case timeType:
return reflect.ValueOf(time.Date(localDate.Year, localDate.Month, localDate.Day, 0, 0, 0, 0, time.Local)), nil
}
case localDateTimeType:
localDateTime := val.Interface().(LocalDateTime)
switch mtype {
case timeType:
return reflect.ValueOf(time.Date(
localDateTime.Date.Year,
localDateTime.Date.Month,
localDateTime.Date.Day,
localDateTime.Time.Hour,
localDateTime.Time.Minute,
localDateTime.Time.Second,
localDateTime.Time.Nanosecond,
time.Local)), nil
}
}
// if this passes for when mtype is reflect.Struct, tval is a time.LocalTime
if !val.Type().ConvertibleTo(mtype) {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to %v", tval, tval, mtype.String())
}
@@ -706,8 +792,15 @@ func (d *Decoder) valueFromToml(mtype reflect.Type, tval interface{}) (reflect.V
}
}
func (d *Decoder) unwrapPointer(mtype reflect.Type, tval interface{}) (reflect.Value, error) {
val, err := d.valueFromToml(mtype.Elem(), tval)
func (d *Decoder) unwrapPointer(mtype reflect.Type, tval interface{}, mval1 *reflect.Value) (reflect.Value, error) {
var melem *reflect.Value
if mval1 != nil && !mval1.IsNil() && mtype.Elem().Kind() == reflect.Struct {
elem := mval1.Elem()
melem = &elem
}
val, err := d.valueFromToml(mtype.Elem(), tval, melem)
if err != nil {
return reflect.ValueOf(nil), err
}
+1
View File
@@ -27,6 +27,7 @@ title = "TOML Marshal Testing"
uint = 5001
bool = true
float = 123.4
float64 = 123.456782132399
int = 5000
string = "Bite me"
date = 1979-05-27T07:32:00Z
+852 -24
View File
@@ -5,6 +5,7 @@ import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
"reflect"
"strings"
"testing"
@@ -55,6 +56,107 @@ Ystrlist = ["Howdy","Hey There"]
String2 = "Three"
`)
var marshalTestToml = []byte(`title = "TOML Marshal Testing"
[basic]
bool = true
date = 1979-05-27T07:32:00Z
float = 123.4
float64 = 123.456782132399
int = 5000
string = "Bite me"
uint = 5001
[basic_lists]
bools = [true,false,true]
dates = [1979-05-27T07:32:00Z,1980-05-27T07:32:00Z]
floats = [12.3,45.6,78.9]
ints = [8001,8001,8002]
strings = ["One","Two","Three"]
uints = [5002,5003]
[basic_map]
one = "one"
two = "two"
[subdoc]
[subdoc.first]
name = "First"
[subdoc.second]
name = "Second"
[[subdoclist]]
name = "List.First"
[[subdoclist]]
name = "List.Second"
[[subdocptrs]]
name = "Second"
`)
var marshalOrderPreserveToml = []byte(`title = "TOML Marshal Testing"
[basic_lists]
floats = [12.3,45.6,78.9]
bools = [true,false,true]
dates = [1979-05-27T07:32:00Z,1980-05-27T07:32:00Z]
ints = [8001,8001,8002]
uints = [5002,5003]
strings = ["One","Two","Three"]
[[subdocptrs]]
name = "Second"
[basic_map]
one = "one"
two = "two"
[subdoc]
[subdoc.second]
name = "Second"
[subdoc.first]
name = "First"
[basic]
uint = 5001
bool = true
float = 123.4
float64 = 123.456782132399
int = 5000
string = "Bite me"
date = 1979-05-27T07:32:00Z
[[subdoclist]]
name = "List.First"
[[subdoclist]]
name = "List.Second"
`)
var mashalOrderPreserveMapToml = []byte(`title = "TOML Marshal Testing"
[basic_map]
one = "one"
two = "two"
[long_map]
a7 = "1"
b3 = "2"
c8 = "3"
d4 = "4"
e6 = "5"
f5 = "6"
g10 = "7"
h1 = "8"
i2 = "9"
j9 = "10"
`)
func TestBasicMarshal(t *testing.T) {
result, err := Marshal(basicTestData)
if err != nil {
@@ -126,10 +228,17 @@ type testDoc struct {
Unexported2 int `toml:"-"`
}
type testMapDoc struct {
Title string `toml:"title"`
BasicMap map[string]string `toml:"basic_map"`
LongMap map[string]string `toml:"long_map"`
}
type testDocBasics struct {
Uint uint `toml:"uint"`
Bool bool `toml:"bool"`
Float float32 `toml:"float"`
Float32 float32 `toml:"float"`
Float64 float64 `toml:"float64"`
Int int `toml:"int"`
String *string `toml:"string"`
Date time.Time `toml:"date"`
@@ -168,7 +277,8 @@ var docData = testDoc{
Basics: testDocBasics{
Bool: true,
Date: time.Date(1979, 5, 27, 7, 32, 0, 0, time.UTC),
Float: 123.4,
Float32: 123.4,
Float64: 123.456782132399,
Int: 5000,
Uint: 5001,
String: &biteMe,
@@ -200,14 +310,33 @@ var docData = testDoc{
SubDocPtrs: []*testSubDoc{&subdoc},
}
var mapTestDoc = testMapDoc{
Title: "TOML Marshal Testing",
BasicMap: map[string]string{
"one": "one",
"two": "two",
},
LongMap: map[string]string{
"h1": "8",
"i2": "9",
"b3": "2",
"d4": "4",
"f5": "6",
"e6": "5",
"a7": "1",
"c8": "3",
"j9": "10",
"g10": "7",
},
}
func TestDocMarshal(t *testing.T) {
result, err := Marshal(docData)
if err != nil {
t.Fatal(err)
}
expected, _ := ioutil.ReadFile("marshal_test.toml")
if !bytes.Equal(result, expected) {
t.Errorf("Bad marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", expected, result)
if !bytes.Equal(result, marshalTestToml) {
t.Errorf("Bad marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", marshalTestToml, result)
}
}
@@ -217,9 +346,29 @@ func TestDocMarshalOrdered(t *testing.T) {
if err != nil {
t.Fatal(err)
}
expected, _ := ioutil.ReadFile("marshal_OrderPreserve_test.toml")
if !bytes.Equal(result.Bytes(), expected) {
t.Errorf("Bad marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", expected, result.Bytes())
if !bytes.Equal(result.Bytes(), marshalOrderPreserveToml) {
t.Errorf("Bad marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", marshalOrderPreserveToml, result.Bytes())
}
}
func TestDocMarshalMaps(t *testing.T) {
result, err := Marshal(mapTestDoc)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(result, mashalOrderPreserveMapToml) {
t.Errorf("Bad marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", mashalOrderPreserveMapToml, result)
}
}
func TestDocMarshalOrderedMaps(t *testing.T) {
var result bytes.Buffer
err := NewEncoder(&result).Order(OrderPreserve).Encode(mapTestDoc)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(result.Bytes(), mashalOrderPreserveMapToml) {
t.Errorf("Bad marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", mashalOrderPreserveMapToml, result.Bytes())
}
}
@@ -228,16 +377,15 @@ func TestDocMarshalPointer(t *testing.T) {
if err != nil {
t.Fatal(err)
}
expected, _ := ioutil.ReadFile("marshal_test.toml")
if !bytes.Equal(result, expected) {
t.Errorf("Bad marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", expected, result)
if !bytes.Equal(result, marshalTestToml) {
t.Errorf("Bad marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", marshalTestToml, result)
}
}
func TestDocUnmarshal(t *testing.T) {
result := testDoc{}
tomlData, _ := ioutil.ReadFile("marshal_test.toml")
err := Unmarshal(tomlData, &result)
err := Unmarshal(marshalTestToml, &result)
expected := docData
if err != nil {
t.Fatal(err)
@@ -250,11 +398,22 @@ func TestDocUnmarshal(t *testing.T) {
}
func TestDocPartialUnmarshal(t *testing.T) {
result := testDocSubs{}
file, err := ioutil.TempFile("", "test-*.toml")
if err != nil {
t.Fatal(err)
}
defer os.Remove(file.Name())
tree, _ := LoadFile("marshal_test.toml")
err = ioutil.WriteFile(file.Name(), marshalTestToml, 0)
if err != nil {
t.Fatal(err)
}
tree, _ := LoadFile(file.Name())
subTree := tree.Get("subdoc").(*Tree)
err := subTree.Unmarshal(&result)
result := testDocSubs{}
err = subTree.Unmarshal(&result)
expected := docData.Subdocs
if err != nil {
t.Fatal(err)
@@ -274,12 +433,36 @@ type tomlTypeCheckTest struct {
func TestTypeChecks(t *testing.T) {
tests := []tomlTypeCheckTest{
{"integer", 2, 0},
{"bool", true, 0},
{"bool", false, 0},
{"int", int(2), 0},
{"int8", int8(2), 0},
{"int16", int16(2), 0},
{"int32", int32(2), 0},
{"int64", int64(2), 0},
{"uint", uint(2), 0},
{"uint8", uint8(2), 0},
{"uint16", uint16(2), 0},
{"uint32", uint32(2), 0},
{"uint64", uint64(2), 0},
{"float32", float32(3.14), 0},
{"float64", float64(3.14), 0},
{"string", "lorem ipsum", 0},
{"time", time.Date(2015, 1, 1, 0, 0, 0, 0, time.UTC), 0},
{"stringlist", []string{"hello", "hi"}, 1},
{"stringlistptr", &[]string{"hello", "hi"}, 1},
{"stringarray", [2]string{"hello", "hi"}, 1},
{"stringarrayptr", &[2]string{"hello", "hi"}, 1},
{"timelist", []time.Time{time.Date(2015, 1, 1, 0, 0, 0, 0, time.UTC)}, 1},
{"timelistptr", &[]time.Time{time.Date(2015, 1, 1, 0, 0, 0, 0, time.UTC)}, 1},
{"timearray", [1]time.Time{time.Date(2015, 1, 1, 0, 0, 0, 0, time.UTC)}, 1},
{"timearrayptr", &[1]time.Time{time.Date(2015, 1, 1, 0, 0, 0, 0, time.UTC)}, 1},
{"objectlist", []tomlTypeCheckTest{}, 2},
{"objectlistptr", &[]tomlTypeCheckTest{}, 2},
{"objectarray", [2]tomlTypeCheckTest{{}, {}}, 2},
{"objectlistptr", &[2]tomlTypeCheckTest{{}, {}}, 2},
{"object", tomlTypeCheckTest{}, 3},
{"objectptr", &tomlTypeCheckTest{}, 3},
}
for _, test := range tests {
@@ -287,8 +470,8 @@ func TestTypeChecks(t *testing.T) {
expected[test.typ] = true
result := []bool{
isPrimitive(reflect.TypeOf(test.item)),
isOtherSlice(reflect.TypeOf(test.item)),
isTreeSlice(reflect.TypeOf(test.item)),
isOtherSequence(reflect.TypeOf(test.item)),
isTreeSequence(reflect.TypeOf(test.item)),
isTree(reflect.TypeOf(test.item)),
}
if !reflect.DeepEqual(expected, result) {
@@ -976,6 +1159,21 @@ func TestMarshalCustomCommented(t *testing.T) {
}
}
func TestMarshalDirectMultilineString(t *testing.T) {
tree := newTree()
tree.SetWithOptions("mykey", SetOptions{
Multiline: true,
}, "my\x11multiline\nstring\ba\tb\fc\rd\"e\\!")
result, err := tree.Marshal()
if err != nil {
t.Fatal("marshal should not error:", err)
}
expected := []byte("mykey = \"\"\"\nmy\\u0011multiline\nstring\\ba\tb\\fc\rd\"e\\!\"\"\"\n")
if !bytes.Equal(result, expected) {
t.Errorf("Bad marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", expected, result)
}
}
var customMultilineTagTestToml = []byte(`int_slice = [
1,
2,
@@ -1054,12 +1252,68 @@ func TestUnmarshalCustomTag(t *testing.T) {
}
func TestUnmarshalMap(t *testing.T) {
m := make(map[string]int)
m["a"] = 1
testToml := []byte(`
a = 1
b = 2
c = 3
`)
var result map[string]int
err := Unmarshal(testToml, &result)
if err != nil {
t.Errorf("Received unexpected error: %s", err)
return
}
err := Unmarshal(basicTestToml, m)
if err.Error() != "Only a pointer to struct can be unmarshaled from TOML" {
t.Fail()
expected := map[string]int{
"a": 1,
"b": 2,
"c": 3,
}
if !reflect.DeepEqual(result, expected) {
t.Errorf("Bad unmarshal: expected %v, got %v", expected, result)
}
}
func TestUnmarshalMapWithTypedKey(t *testing.T) {
testToml := []byte(`
a = 1
b = 2
c = 3
`)
type letter string
var result map[letter]int
err := Unmarshal(testToml, &result)
if err != nil {
t.Errorf("Received unexpected error: %s", err)
return
}
expected := map[letter]int{
"a": 1,
"b": 2,
"c": 3,
}
if !reflect.DeepEqual(result, expected) {
t.Errorf("Bad unmarshal: expected %v, got %v", expected, result)
}
}
func TestUnmarshalNonPointer(t *testing.T) {
a := 1
err := Unmarshal([]byte{}, a)
if err == nil {
t.Fatal("unmarshal should err when given a non pointer")
}
}
func TestUnmarshalInvalidPointerKind(t *testing.T) {
a := 1
err := Unmarshal([]byte{}, &a)
if err == nil {
t.Fatal("unmarshal should err when given an invalid pointer type")
}
}
@@ -1312,3 +1566,577 @@ func TestUnmarshalDefaultFailureUnsupported(t *testing.T) {
t.Fatal("should error")
}
}
func TestUnmarshalNestedAnonymousStructs(t *testing.T) {
type Nested struct {
Value string `toml:"nested_field"`
}
type Deep struct {
Nested
}
type Document struct {
Deep
Value string `toml:"own_field"`
}
var doc Document
err := Unmarshal([]byte(`nested_field = "nested value"`+"\n"+`own_field = "own value"`), &doc)
if err != nil {
t.Fatal("should not error")
}
if doc.Value != "own value" || doc.Nested.Value != "nested value" {
t.Fatal("unexpected values")
}
}
func TestUnmarshalNestedAnonymousStructs_Controversial(t *testing.T) {
type Nested struct {
Value string `toml:"nested"`
}
type Deep struct {
Nested
}
type Document struct {
Deep
Value string `toml:"own"`
}
var doc Document
err := Unmarshal([]byte(`nested = "nested value"`+"\n"+`own = "own value"`), &doc)
if err == nil {
t.Fatal("should error")
}
}
type unexportedFieldPreservationTest struct {
Exported string `toml:"exported"`
unexported string
Nested1 unexportedFieldPreservationTestNested `toml:"nested1"`
Nested2 *unexportedFieldPreservationTestNested `toml:"nested2"`
Nested3 *unexportedFieldPreservationTestNested `toml:"nested3"`
Slice1 []unexportedFieldPreservationTestNested `toml:"slice1"`
Slice2 []*unexportedFieldPreservationTestNested `toml:"slice2"`
}
type unexportedFieldPreservationTestNested struct {
Exported1 string `toml:"exported1"`
unexported1 string
}
func TestUnmarshalPreservesUnexportedFields(t *testing.T) {
toml := `
exported = "visible"
unexported = "ignored"
[nested1]
exported1 = "visible1"
unexported1 = "ignored1"
[nested2]
exported1 = "visible2"
unexported1 = "ignored2"
[nested3]
exported1 = "visible3"
unexported1 = "ignored3"
[[slice1]]
exported1 = "visible3"
[[slice1]]
exported1 = "visible4"
[[slice2]]
exported1 = "visible5"
`
t.Run("unexported field should not be set from toml", func(t *testing.T) {
var actual unexportedFieldPreservationTest
err := Unmarshal([]byte(toml), &actual)
if err != nil {
t.Fatal("did not expect an error")
}
expect := unexportedFieldPreservationTest{
Exported: "visible",
unexported: "",
Nested1: unexportedFieldPreservationTestNested{"visible1", ""},
Nested2: &unexportedFieldPreservationTestNested{"visible2", ""},
Nested3: &unexportedFieldPreservationTestNested{"visible3", ""},
Slice1: []unexportedFieldPreservationTestNested{
{Exported1: "visible3"},
{Exported1: "visible4"},
},
Slice2: []*unexportedFieldPreservationTestNested{
{Exported1: "visible5"},
},
}
if !reflect.DeepEqual(actual, expect) {
t.Fatalf("%+v did not equal %+v", actual, expect)
}
})
t.Run("unexported field should be preserved", func(t *testing.T) {
actual := unexportedFieldPreservationTest{
Exported: "foo",
unexported: "bar",
Nested1: unexportedFieldPreservationTestNested{"baz", "bax"},
Nested2: nil,
Nested3: &unexportedFieldPreservationTestNested{"baz", "bax"},
}
err := Unmarshal([]byte(toml), &actual)
if err != nil {
t.Fatal("did not expect an error")
}
expect := unexportedFieldPreservationTest{
Exported: "visible",
unexported: "bar",
Nested1: unexportedFieldPreservationTestNested{"visible1", "bax"},
Nested2: &unexportedFieldPreservationTestNested{"visible2", ""},
Nested3: &unexportedFieldPreservationTestNested{"visible3", "bax"},
Slice1: []unexportedFieldPreservationTestNested{
{Exported1: "visible3"},
{Exported1: "visible4"},
},
Slice2: []*unexportedFieldPreservationTestNested{
{Exported1: "visible5"},
},
}
if !reflect.DeepEqual(actual, expect) {
t.Fatalf("%+v did not equal %+v", actual, expect)
}
})
}
func TestTreeMarshal(t *testing.T) {
cases := [][]byte{
basicTestToml,
marshalTestToml,
emptyTestToml,
pointerTestToml,
}
for _, expected := range cases {
t.Run("", func(t *testing.T) {
tree, err := LoadBytes(expected)
if err != nil {
t.Fatal(err)
}
result, err := tree.Marshal()
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(result, expected) {
t.Errorf("Bad marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", expected, result)
}
})
}
}
func TestMarshalArrays(t *testing.T) {
cases := []struct {
Data interface{}
Expected string
}{
{
Data: struct {
XY [2]int
}{
XY: [2]int{1, 2},
},
Expected: `XY = [1,2]
`,
},
{
Data: struct {
XY [1][2]int
}{
XY: [1][2]int{{1, 2}},
},
Expected: `XY = [[1,2]]
`,
},
{
Data: struct {
XY [1][]int
}{
XY: [1][]int{{1, 2}},
},
Expected: `XY = [[1,2]]
`,
},
{
Data: struct {
XY [][2]int
}{
XY: [][2]int{{1, 2}},
},
Expected: `XY = [[1,2]]
`,
},
}
for _, tc := range cases {
t.Run("", func(t *testing.T) {
result, err := Marshal(tc.Data)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(result, []byte(tc.Expected)) {
t.Errorf("Bad marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", []byte(tc.Expected), result)
}
})
}
}
func TestUnmarshalLocalDate(t *testing.T) {
t.Run("ToLocalDate", func(t *testing.T) {
type dateStruct struct {
Date LocalDate
}
toml := `date = 1979-05-27`
var obj dateStruct
err := Unmarshal([]byte(toml), &obj)
if err != nil {
t.Fatal(err)
}
if obj.Date.Year != 1979 {
t.Errorf("expected year 1979, got %d", obj.Date.Year)
}
if obj.Date.Month != 5 {
t.Errorf("expected month 5, got %d", obj.Date.Month)
}
if obj.Date.Day != 27 {
t.Errorf("expected day 27, got %d", obj.Date.Day)
}
})
t.Run("ToLocalDate", func(t *testing.T) {
type dateStruct struct {
Date time.Time
}
toml := `date = 1979-05-27`
var obj dateStruct
err := Unmarshal([]byte(toml), &obj)
if err != nil {
t.Fatal(err)
}
if obj.Date.Year() != 1979 {
t.Errorf("expected year 1979, got %d", obj.Date.Year())
}
if obj.Date.Month() != 5 {
t.Errorf("expected month 5, got %d", obj.Date.Month())
}
if obj.Date.Day() != 27 {
t.Errorf("expected day 27, got %d", obj.Date.Day())
}
})
}
func TestMarshalLocalDate(t *testing.T) {
type dateStruct struct {
Date LocalDate
}
obj := dateStruct{Date: LocalDate{
Year: 1979,
Month: 5,
Day: 27,
}}
b, err := Marshal(obj)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
got := string(b)
expected := `Date = 1979-05-27
`
if got != expected {
t.Errorf("expected '%s', got '%s'", expected, got)
}
}
func TestUnmarshalLocalDateTime(t *testing.T) {
examples := []struct {
name string
in string
out LocalDateTime
}{
{
name: "normal",
in: "1979-05-27T07:32:00",
out: LocalDateTime{
Date: LocalDate{
Year: 1979,
Month: 5,
Day: 27,
},
Time: LocalTime{
Hour: 7,
Minute: 32,
Second: 0,
Nanosecond: 0,
},
}},
{
name: "with nanoseconds",
in: "1979-05-27T00:32:00.999999",
out: LocalDateTime{
Date: LocalDate{
Year: 1979,
Month: 5,
Day: 27,
},
Time: LocalTime{
Hour: 0,
Minute: 32,
Second: 0,
Nanosecond: 999999000,
},
},
},
}
for i, example := range examples {
toml := fmt.Sprintf(`date = %s`, example.in)
t.Run(fmt.Sprintf("ToLocalDateTime_%d_%s", i, example.name), func(t *testing.T) {
type dateStruct struct {
Date LocalDateTime
}
var obj dateStruct
err := Unmarshal([]byte(toml), &obj)
if err != nil {
t.Fatal(err)
}
if obj.Date != example.out {
t.Errorf("expected '%s', got '%s'", example.out, obj.Date)
}
})
t.Run(fmt.Sprintf("ToTime_%d_%s", i, example.name), func(t *testing.T) {
type dateStruct struct {
Date time.Time
}
var obj dateStruct
err := Unmarshal([]byte(toml), &obj)
if err != nil {
t.Fatal(err)
}
if obj.Date.Year() != example.out.Date.Year {
t.Errorf("expected year %d, got %d", example.out.Date.Year, obj.Date.Year())
}
if obj.Date.Month() != example.out.Date.Month {
t.Errorf("expected month %d, got %d", example.out.Date.Month, obj.Date.Month())
}
if obj.Date.Day() != example.out.Date.Day {
t.Errorf("expected day %d, got %d", example.out.Date.Day, obj.Date.Day())
}
if obj.Date.Hour() != example.out.Time.Hour {
t.Errorf("expected hour %d, got %d", example.out.Time.Hour, obj.Date.Hour())
}
if obj.Date.Minute() != example.out.Time.Minute {
t.Errorf("expected minute %d, got %d", example.out.Time.Minute, obj.Date.Minute())
}
if obj.Date.Second() != example.out.Time.Second {
t.Errorf("expected second %d, got %d", example.out.Time.Second, obj.Date.Second())
}
if obj.Date.Nanosecond() != example.out.Time.Nanosecond {
t.Errorf("expected nanoseconds %d, got %d", example.out.Time.Nanosecond, obj.Date.Nanosecond())
}
})
}
}
func TestMarshalLocalDateTime(t *testing.T) {
type dateStruct struct {
DateTime LocalDateTime
}
examples := []struct {
name string
in LocalDateTime
out string
}{
{
name: "normal",
out: "DateTime = 1979-05-27T07:32:00\n",
in: LocalDateTime{
Date: LocalDate{
Year: 1979,
Month: 5,
Day: 27,
},
Time: LocalTime{
Hour: 7,
Minute: 32,
Second: 0,
Nanosecond: 0,
},
}},
{
name: "with nanoseconds",
out: "DateTime = 1979-05-27T00:32:00.999999000\n",
in: LocalDateTime{
Date: LocalDate{
Year: 1979,
Month: 5,
Day: 27,
},
Time: LocalTime{
Hour: 0,
Minute: 32,
Second: 0,
Nanosecond: 999999000,
},
},
},
}
for i, example := range examples {
t.Run(fmt.Sprintf("%d_%s", i, example.name), func(t *testing.T) {
obj := dateStruct{
DateTime: example.in,
}
b, err := Marshal(obj)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
got := string(b)
if got != example.out {
t.Errorf("expected '%s', got '%s'", example.out, got)
}
})
}
}
func TestUnmarshalLocalTime(t *testing.T) {
examples := []struct {
name string
in string
out LocalTime
}{
{
name: "normal",
in: "07:32:00",
out: LocalTime{
Hour: 7,
Minute: 32,
Second: 0,
Nanosecond: 0,
},
},
{
name: "with nanoseconds",
in: "00:32:00.999999",
out: LocalTime{
Hour: 0,
Minute: 32,
Second: 0,
Nanosecond: 999999000,
},
},
}
for i, example := range examples {
toml := fmt.Sprintf(`Time = %s`, example.in)
t.Run(fmt.Sprintf("ToLocalTime_%d_%s", i, example.name), func(t *testing.T) {
type dateStruct struct {
Time LocalTime
}
var obj dateStruct
err := Unmarshal([]byte(toml), &obj)
if err != nil {
t.Fatal(err)
}
if obj.Time != example.out {
t.Errorf("expected '%s', got '%s'", example.out, obj.Time)
}
})
}
}
func TestMarshalLocalTime(t *testing.T) {
type timeStruct struct {
Time LocalTime
}
examples := []struct {
name string
in LocalTime
out string
}{
{
name: "normal",
out: "Time = 07:32:00\n",
in: LocalTime{
Hour: 7,
Minute: 32,
Second: 0,
Nanosecond: 0,
}},
{
name: "with nanoseconds",
out: "Time = 00:32:00.999999000\n",
in: LocalTime{
Hour: 0,
Minute: 32,
Second: 0,
Nanosecond: 999999000,
},
},
}
for i, example := range examples {
t.Run(fmt.Sprintf("%d_%s", i, example.name), func(t *testing.T) {
obj := timeStruct{
Time: example.in,
}
b, err := Marshal(obj)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
got := string(b)
if got != example.out {
t.Errorf("expected '%s', got '%s'", example.out, got)
}
})
}
}
+1
View File
@@ -4,6 +4,7 @@ title = "TOML Marshal Testing"
bool = true
date = 1979-05-27T07:32:00Z
float = 123.4
float64 = 123.456782132399
int = 5000
string = "Bite me"
uint = 5001
+43 -6
View File
@@ -313,7 +313,41 @@ func (p *tomlParser) parseRvalue() interface{} {
}
return val
case tokenDate:
val, err := time.ParseInLocation(time.RFC3339Nano, tok.val, time.UTC)
layout := time.RFC3339Nano
if !strings.Contains(tok.val, "T") {
layout = strings.Replace(layout, "T", " ", 1)
}
val, err := time.ParseInLocation(layout, tok.val, time.UTC)
if err != nil {
p.raiseError(tok, "%s", err)
}
return val
case tokenLocalDate:
v := strings.Replace(tok.val, " ", "T", -1)
isDateTime := false
isTime := false
for _, c := range v {
if c == 'T' || c == 't' {
isDateTime = true
break
}
if c == ':' {
isTime = true
break
}
}
var val interface{}
var err error
if isDateTime {
val, err = ParseLocalDateTime(v)
} else if isTime {
val, err = ParseLocalTime(v)
} else {
val, err = ParseLocalDate(v)
}
if err != nil {
p.raiseError(tok, "%s", err)
}
@@ -356,12 +390,15 @@ Loop:
}
key := p.getToken()
p.assume(tokenEqual)
value := p.parseRvalue()
tree.Set(key.val, value)
case tokenComma:
if previous == nil {
p.raiseError(follow, "inline table cannot start with a comma")
parsedKey, err := parseKey(key.val)
if err != nil {
p.raiseError(key, "invalid key: %s", err)
}
value := p.parseRvalue()
tree.SetPath(parsedKey, value)
case tokenComma:
if tokenIsComma(previous) {
p.raiseError(follow, "need field between two commas in inline table")
}
+144 -2
View File
@@ -197,7 +197,7 @@ func TestFloatsWithExponents(t *testing.T) {
tree, err := Load("a = 5e+22\nb = 5E+22\nc = -5e+22\nd = -5e-22\ne = 6.626e-34")
assertTree(t, tree, err, map[string]interface{}{
"a": float64(5e+22),
"b": float64(5E+22),
"b": float64(5e+22),
"c": float64(-5e+22),
"d": float64(-5e-22),
"e": float64(6.626e-34),
@@ -225,6 +225,77 @@ func TestDateNano(t *testing.T) {
})
}
func TestLocalDateTime(t *testing.T) {
tree, err := Load("a = 1979-05-27T07:32:00")
assertTree(t, tree, err, map[string]interface{}{
"a": LocalDateTime{
Date: LocalDate{
Year: 1979,
Month: 5,
Day: 27,
},
Time: LocalTime{
Hour: 7,
Minute: 32,
Second: 0,
Nanosecond: 0,
}},
})
}
func TestLocalDateTimeNano(t *testing.T) {
tree, err := Load("a = 1979-05-27T07:32:00.999999")
assertTree(t, tree, err, map[string]interface{}{
"a": LocalDateTime{
Date: LocalDate{
Year: 1979,
Month: 5,
Day: 27,
},
Time: LocalTime{
Hour: 7,
Minute: 32,
Second: 0,
Nanosecond: 999999000,
}},
})
}
func TestLocalDate(t *testing.T) {
tree, err := Load("a = 1979-05-27")
assertTree(t, tree, err, map[string]interface{}{
"a": LocalDate{
Year: 1979,
Month: 5,
Day: 27,
},
})
}
func TestLocalTime(t *testing.T) {
tree, err := Load("a = 07:32:00")
assertTree(t, tree, err, map[string]interface{}{
"a": LocalTime{
Hour: 7,
Minute: 32,
Second: 0,
Nanosecond: 0,
},
})
}
func TestLocalTimeNano(t *testing.T) {
tree, err := Load("a = 00:32:00.999999")
assertTree(t, tree, err, map[string]interface{}{
"a": LocalTime{
Hour: 0,
Minute: 32,
Second: 0,
Nanosecond: 999999000,
},
})
}
func TestSimpleString(t *testing.T) {
tree, err := Load("a = \"hello world\"")
assertTree(t, tree, err, map[string]interface{}{
@@ -525,6 +596,33 @@ point = { x = 1, y = 2 }`)
})
}
func TestInlineGroupBareKeysUnderscore(t *testing.T) {
tree, err := Load(`foo = { _bar = "buz" }`)
assertTree(t, tree, err, map[string]interface{}{
"foo": map[string]interface{}{
"_bar": "buz",
},
})
}
func TestInlineGroupBareKeysDash(t *testing.T) {
tree, err := Load(`foo = { -bar = "buz" }`)
assertTree(t, tree, err, map[string]interface{}{
"foo": map[string]interface{}{
"-bar": "buz",
},
})
}
func TestInlineGroupKeyQuoted(t *testing.T) {
tree, err := Load(`foo = { "bar" = "buz" }`)
assertTree(t, tree, err, map[string]interface{}{
"foo": map[string]interface{}{
"bar": "buz",
},
})
}
func TestExampleInlineGroupInArray(t *testing.T) {
tree, err := Load(`points = [{ x = 1, y = 2 }]`)
assertTree(t, tree, err, map[string]interface{}{
@@ -553,7 +651,7 @@ func TestInlineTableCommaExpected(t *testing.T) {
func TestInlineTableCommaStart(t *testing.T) {
_, err := Load("foo = {, hello = 53}")
if err.Error() != "(1, 8): inline table cannot start with a comma" {
if err.Error() != "(1, 8): unexpected token type in inline table: keys cannot contain , character" {
t.Error("Bad error message:", err.Error())
}
}
@@ -909,6 +1007,13 @@ func TestMapKeyIsNum(t *testing.T) {
}
}
func TestInvalidKeyInlineTable(t *testing.T) {
_, err := Load("table={invalid..key = 1}")
if err.Error() != "(1, 8): invalid key: expecting key part after dot" {
t.Error("Bad error message:", err.Error())
}
}
func TestDottedKeys(t *testing.T) {
tree, err := Load(`
name = "Orange"
@@ -937,3 +1042,40 @@ func TestInvalidDottedKeyEmptyGroup(t *testing.T) {
t.Fatalf("invalid error message: %s", err)
}
}
func TestAccidentalNewlines(t *testing.T) {
expected := "The quick brown fox jumps over the lazy dog."
tree, err := Load(`str1 = "The quick brown fox jumps over the lazy dog."
str2 = """
The quick brown \
fox jumps over \
the lazy dog."""
str3 = """\
The quick brown \
fox jumps over \
the lazy dog.\
"""`)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
got := tree.Get("str1")
if got != expected {
t.Errorf("expected '%s', got '%s'", expected, got)
}
got = tree.Get("str2")
if got != expected {
t.Errorf("expected '%s', got '%s'", expected, got)
}
got = tree.Get("str3")
if got != expected {
t.Errorf("expected '%s', got '%s'", expected, got)
}
}
+3 -10
View File
@@ -2,7 +2,6 @@ package toml
import (
"fmt"
"strconv"
"unicode"
)
@@ -35,6 +34,7 @@ const (
tokenDoubleLeftBracket
tokenDoubleRightBracket
tokenDate
tokenLocalDate
tokenKeyGroup
tokenKeyGroupArray
tokenComma
@@ -68,7 +68,8 @@ var tokenTypeNames = []string{
")",
"]]",
"[[",
"Date",
"LocalDate",
"LocalDate",
"KeyGroup",
"KeyGroupArray",
",",
@@ -95,14 +96,6 @@ func (tt tokenType) String() string {
return "Unknown"
}
func (t token) Int() int {
if result, err := strconv.Atoi(t.val); err != nil {
panic(err)
} else {
return result
}
}
func (t token) String() string {
switch t.typ {
case tokenEOF:
+2 -1
View File
@@ -25,7 +25,8 @@ func TestTokenStringer(t *testing.T) {
{tokenRightParen, ")"},
{tokenDoubleLeftBracket, "]]"},
{tokenDoubleRightBracket, "[["},
{tokenDate, "Date"},
{tokenDate, "LocalDate"},
{tokenLocalDate, "LocalDate"},
{tokenKeyGroup, "KeyGroup"},
{tokenKeyGroupArray, "KeyGroupArray"},
{tokenComma, ","},
+1 -1
View File
@@ -275,7 +275,7 @@ func (t *Tree) Delete(key string) error {
return t.DeletePath(keys)
}
// Delete removes a key from the tree.
// DeletePath removes a key from the tree.
// Keys is an array of path elements (e.g. {"a","b","c"}).
func (t *Tree) DeletePath(keys []string) error {
keyLen := len(keys)
+51 -10
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"io"
"math"
"math/big"
"reflect"
"sort"
"strconv"
@@ -106,12 +107,20 @@ func tomlValueStringRepresentation(v interface{}, indent string, arraysOneElemen
case int64:
return strconv.FormatInt(value, 10), nil
case float64:
// Ensure a round float does contain a decimal point. Otherwise feeding
// the output back to the parser would convert to an integer.
if math.Trunc(value) == value {
return strings.ToLower(strconv.FormatFloat(value, 'f', 1, 32)), nil
// Default bit length is full 64
bits := 64
// Float panics if nan is used
if !math.IsNaN(value) {
// if 32 bit accuracy is enough to exactly show, use 32
_, acc := big.NewFloat(value).Float32()
if acc == big.Exact {
bits = 32
}
return strings.ToLower(strconv.FormatFloat(value, 'f', -1, 32)), nil
}
if math.Trunc(value) == value {
return strings.ToLower(strconv.FormatFloat(value, 'f', 1, bits)), nil
}
return strings.ToLower(strconv.FormatFloat(value, 'f', -1, bits)), nil
case string:
if tv.multiline {
return "\"\"\"\n" + encodeMultilineTomlString(value) + "\"\"\"", nil
@@ -127,6 +136,12 @@ func tomlValueStringRepresentation(v interface{}, indent string, arraysOneElemen
return "false", nil
case time.Time:
return value.Format(time.RFC3339), nil
case LocalDate:
return value.String(), nil
case LocalDateTime:
return value.String(), nil
case LocalTime:
return value.String(), nil
case nil:
return "", nil
}
@@ -259,7 +274,7 @@ func (t *Tree) writeTo(w io.Writer, indent, keyspace string, bytesCount int64, a
}
func (t *Tree) writeToOrdered(w io.Writer, indent, keyspace string, bytesCount int64, arraysOneElementPerLine bool, ord marshalOrder) (int64, error) {
orderedVals := make([]sortNode, 0)
var orderedVals []sortNode
switch ord {
case OrderPreserve:
@@ -354,7 +369,8 @@ func (t *Tree) writeToOrdered(w io.Writer, indent, keyspace string, bytesCount i
if v.commented {
commented = "# "
}
writtenBytesCount, err := writeStrings(w, indent, commented, k, " = ", repr, "\n")
quotedKey := quoteKeyIfNeeded(k)
writtenBytesCount, err := writeStrings(w, indent, commented, quotedKey, " = ", repr, "\n")
bytesCount += int64(writtenBytesCount)
if err != nil {
return bytesCount, err
@@ -365,6 +381,32 @@ func (t *Tree) writeToOrdered(w io.Writer, indent, keyspace string, bytesCount i
return bytesCount, nil
}
// quote a key if it does not fit the bare key format (A-Za-z0-9_-)
// quoted keys use the same rules as strings
func quoteKeyIfNeeded(k string) string {
// when encoding a map with the 'quoteMapKeys' option enabled, the tree will contain
// keys that have already been quoted.
// not an ideal situation, but good enough of a stop gap.
if len(k) >= 2 && k[0] == '"' && k[len(k)-1] == '"' {
return k
}
isBare := true
for _, r := range k {
if !isValidBareChar(r) {
isBare = false
break
}
}
if isBare {
return k
}
return quoteKey(k)
}
func quoteKey(k string) string {
return "\"" + encodeTomlString(k) + "\""
}
func writeStrings(w io.Writer, s ...string) (int, error) {
var n int
for i := range s {
@@ -387,12 +429,11 @@ func (t *Tree) WriteTo(w io.Writer) (int64, error) {
// Output spans multiple lines, and is suitable for ingest by a TOML parser.
// If the conversion cannot be performed, ToString returns a non-nil error.
func (t *Tree) ToTomlString() (string, error) {
var buf bytes.Buffer
_, err := t.WriteTo(&buf)
b, err := t.Marshal()
if err != nil {
return "", err
}
return buf.String(), nil
return string(b), nil
}
// String generates a human-readable representation of the current tree.
+24
View File
@@ -327,6 +327,30 @@ c = nan`
}
}
func TestIssue290(t *testing.T) {
tomlString :=
`[table]
"127.0.0.1" = "value"
"127.0.0.1:8028" = "value"
"character encoding" = "value"
"ʎǝʞ" = "value"`
t1, err := Load(tomlString)
if err != nil {
t.Fatal("load err:", err)
}
s, err := t1.ToTomlString()
if err != nil {
t.Fatal("ToTomlString err:", err)
}
_, err = Load(s)
if err != nil {
t.Fatal("reload err:", err)
}
}
func BenchmarkTreeToTomlString(b *testing.B) {
toml, err := Load(sampleHard)
if err != nil {