Compare commits

...

23 Commits

Author SHA1 Message Date
Alan Murtagh c01d1270ff Multiline Marshal tag (#221)
The new multiline tag works just like the existing 'commented' tag (i.e.
`multiline:"true"`), and tells go-toml to marshal the value as a
multi-line string. The tag currently has no impact on any non-string
fields.
2018-06-05 13:47:19 -07:00
Cameron Moore 66540cf1fc Go 1.10 support (#223)
* Update Travis CI to use latest Go releases

* Fix go vet issues for Go 1.10

Starting in Go 1.10, the `go test` command now automatically runs `go
vet`. This commit fixes two issues flagged by vet that cause `go test`
to fail.

* Fix gofmt issue for Go 1.10

Go 1.10 introduced a small formatting change with comments in empty
multiline slice definitions.  This commit attempts to move the offending
comment in such a way that all version of gofmt will agree on its
location.

* Remove go-vet from test.sh

Starting in Go 1.10, the `go test` command automatically runs `go vet`,
so we don't need to run `go vet` explicitly from within test.sh.

Fixes #222
2018-03-23 11:52:43 -07:00
Chris 05bcc0fb0d Make multi-line arrays always use trailing commas (#217)
This makes ArraysWithOneElementPerLine output arrays with commas after every element.

```
A = [1,2,3]
```

Now becomes:

```
A = [
  1,
  2,
  3,
]
```
2018-02-28 15:36:31 -08:00
Thomas Pelletier acdc450948 Fix backward incompatibility for Set* methods (#213)
Patch #185 introduced a backward incompatibility by changing the arguments
of the `Set*` methods on `Tree`.

This change restores the arguments to what they previous were, and
introduces `SetWithComment` and `SetPathWithComment` to perform the same
action.
2018-01-18 14:54:55 -08:00
Jelte Fennema 778c285afa Add support for special float values (inf and nan) (#210) 2018-01-18 14:10:55 -08:00
Jelte Fennema a1e8a8d702 Unmarshal into custom types and error on overflows (#209)
This fixes two unmarshalling issues:

1. Unmarshalling into a custom integer/float type (e.g. `time.Duration`).
2. Checks for overflows happen before unmarshalling, erroring if an overflow
would happen.

Apart from this it also reduces code duplication a bit.
2018-01-18 14:08:34 -08:00
Jelte Fennema 03c6bf4172 Actually show the error message from an Error token (#208) 2018-01-18 14:03:34 -08:00
Thomas Pelletier a1b12e18b7 Fix false positive when running test.sh (#212)
Patch #193 introduced a regression in the toml-tests examples, but it was
never caught because test.sh was exiting with a zero status code, even
though the tests failed. This is because of the `|tee` operation when
invoking toml-test, without setting the pipefail option, reporting the
status code of `tee` instead of `toml-test`.
2018-01-18 14:02:09 -08:00
Kazuyoshi Kato 4874e8477b Fix parsing of single quoted keys (#201)
Patch #193 doesn't work correctly because that must be handled by the
lexer, and `parseKey()` must not handle escape sequences.

Ref #61
2018-01-18 13:52:12 -08:00
Thomas Pelletier 9bf0212445 Bump go versions (#211) 2018-01-15 16:08:35 -08:00
Thomas Pelletier 0131db6d73 Lint (#206) 2017-12-22 12:45:48 +01:00
Thomas Pelletier 861c4734ac Support for hex, oct, and bin integers (#205)
Add support for non-decimal integers. At the time of writing, this is an
unreleased backward-compatible feature of TOML:

```
  Non-negative integer values may also be expressed in hexadecimal, octal, or
  binary. In these formats, leading zeros are allowed (after the prefix). Hex
  values are case insensitive. Underscores are allowed between digits (but
  not between the prefix and the value).

  # hexadecimal with prefix `0x`
  hex1 = 0xDEADBEEF
  hex2 = 0xdeadbeef
  hex3 = 0xdead_beef

  # octal with prefix `0o`
  oct1 = 0o01234567
  oct2 = 0o755 # useful for Unix file permissions

  # binary with prefix `0b`
  bin1 = 0b11010110
```

Fixes #204
2017-12-22 12:24:26 +01:00
Thomas Pelletier b8b5e76965 Add Encoder opt to emit arrays on multiple lines (#203)
A new Encoder option emits arrays with more than one line on multiple lines.
This is off by default and toggled with `ArraysWithOneElementPerLine`.

For example:

```
A = [1,2,3]
```

Becomes:

```
A = [
  1,
  2,
  3
]
```

Fixes #200
2017-12-18 14:57:16 +01:00
Robert Günzler 4e9e0ee19b Add Encoder/Decoder types (#192)
Usage is similar to the stdlibs JSON encoder/decoder but I tried to
leave the general structure of the code the same.

Main motivation was to support encoding/decoding options to allow
encoding string-type map keys as quoted TOML keys.
This was implemented on the Encoder with QuoteMapKeys(bool).

> The TOML spec supports using UTF-8 strings as keys.
> https://github.com/toml-lang/toml/blob/master/versions/en/toml-v0.4.0.md#table
2017-10-24 14:10:38 -07:00
Thomas Pelletier 8c31c2ec65 Fix fuzz (#199)
Fix fuzzer (LoadString does not exist), and mention it in the README.
2017-10-21 19:23:38 -07:00
Thomas Pelletier 6d858869d3 Describe versioning policy in README (#198) 2017-10-21 19:08:12 -07:00
Kazuyoshi Kato 1916042ba2 Add fuzz.sh to do fuzzing with go-fuzz (#194)
Fixes #181
2017-10-21 16:37:53 -07:00
Kazuyoshi Kato a410399d2c Support single quoted keys (#193)
Fixes #61
2017-10-21 23:14:36 +00:00
Kazuyoshi Kato 878c11e70e Unmarshal should report a type mismatch as an error (#196)
Fixes #186
2017-10-21 15:29:03 -07:00
Kazuyoshi Kato 19ece5dc77 Fix typos (#195)
Most of them are caught by Go Report Card.
https://goreportcard.com/report/github.com/pelletier/go-toml
2017-10-21 15:26:06 -07:00
Maxime Le Conte des Floris d01db88be9 Fix example code in README (#197) 2017-10-21 15:24:36 -07:00
Thomas Pelletier 2009e44b6f Improve doc for un/marshal functions (#189) 2017-10-01 15:47:47 -07:00
Yvonnick Esnault 690dbc9ee7 Comment annotation for Marshal (#185) 2017-10-01 15:05:24 -07:00
26 changed files with 1095 additions and 254 deletions
+1
View File
@@ -1 +1,2 @@
test_program/test_program_bin test_program/test_program_bin
fuzz/
+3 -3
View File
@@ -1,9 +1,9 @@
sudo: false sudo: false
language: go language: go
go: go:
- 1.7.6 - 1.8.x
- 1.8.3 - 1.9.x
- 1.9 - 1.10.x
- tip - tip
matrix: matrix:
allow_failures: allow_failures:
+15 -3
View File
@@ -57,9 +57,9 @@ type Config struct {
} }
doc := []byte(` doc := []byte(`
[postgres] [Postgres]
user = "pelletier" User = "pelletier"
password = "mypassword"`) Password = "mypassword"`)
config := Config{} config := Config{}
toml.Unmarshal(doc, &config) toml.Unmarshal(doc, &config)
@@ -114,6 +114,18 @@ You have to make sure two kind of tests run:
You can run both of them using `./test.sh`. You can run both of them using `./test.sh`.
### Fuzzing
The script `./fuzz.sh` is available to
run [go-fuzz](https://github.com/dvyukov/go-fuzz) on go-toml.
## Versioning
Go-toml follows [Semantic Versioning](http://semver.org/). The supported version
of [TOML](https://github.com/toml-lang/toml) is indicated at the beginning of
this document. The last two major versions of Go are supported
(see [Go Release Policy](https://golang.org/doc/devel/release.html#policy)).
## License ## License
The MIT License (MIT). Read [LICENSE](LICENSE). The MIT License (MIT). Read [LICENSE](LICENSE).
+6 -7
View File
@@ -17,13 +17,12 @@ import (
func main() { func main() {
flag.Usage = func() { flag.Usage = func() {
fmt.Fprintln(os.Stderr, `tomljson can be used in two ways: fmt.Fprintln(os.Stderr, "tomljson can be used in two ways:")
Writing to STDIN and reading from STDOUT: fmt.Fprintln(os.Stderr, "Writing to STDIN and reading from STDOUT:")
cat file.toml | tomljson > file.json fmt.Fprintln(os.Stderr, " cat file.toml | tomljson > file.json")
fmt.Fprintln(os.Stderr, "")
Reading from a file name: fmt.Fprintln(os.Stderr, "Reading from a file name:")
tomljson file.toml fmt.Fprintln(os.Stderr, " tomljson file.toml")
`)
} }
flag.Parse() flag.Parse()
os.Exit(processMain(flag.Args(), os.Stdin, os.Stdout, os.Stderr)) os.Exit(processMain(flag.Args(), os.Stdin, os.Stdout, os.Stderr))
+8 -9
View File
@@ -17,15 +17,14 @@ import (
func main() { func main() {
flag.Usage = func() { flag.Usage = func() {
fmt.Fprintln(os.Stderr, `tomll can be used in two ways: fmt.Fprintln(os.Stderr, "tomll can be used in two ways:")
Writing to STDIN and reading from STDOUT: fmt.Fprintln(os.Stderr, "Writing to STDIN and reading from STDOUT:")
cat file.toml | tomll > file.toml fmt.Fprintln(os.Stderr, " cat file.toml | tomll > file.toml")
fmt.Fprintln(os.Stderr, "")
Reading and updating a list of files: fmt.Fprintln(os.Stderr, "Reading and updating a list of files:")
tomll a.toml b.toml c.toml fmt.Fprintln(os.Stderr, " tomll a.toml b.toml c.toml")
fmt.Fprintln(os.Stderr, "")
When given a list of files, tomll will modify all files in place without asking. fmt.Fprintln(os.Stderr, "When given a list of files, tomll will modify all files in place without asking.")
`)
} }
flag.Parse() flag.Parse()
// read from stdin and print to stdout // read from stdin and print to stdout
+1 -1
View File
@@ -17,7 +17,7 @@
// JSONPath-like queries // JSONPath-like queries
// //
// The package github.com/pelletier/go-toml/query implements a system // The package github.com/pelletier/go-toml/query implements a system
// similar to JSONPath to quickly retrive elements of a TOML document using a // similar to JSONPath to quickly retrieve elements of a TOML document using a
// single expression. See the package documentation for more information. // single expression. See the package documentation for more information.
// //
package toml package toml
+7 -2
View File
@@ -61,19 +61,24 @@ func ExampleMarshal() {
type Postgres struct { type Postgres struct {
User string `toml:"user"` User string `toml:"user"`
Password string `toml:"password"` Password string `toml:"password"`
Database string `toml:"db" commented:"true" comment:"not used anymore"`
} }
type Config struct { type Config struct {
Postgres Postgres `toml:"postgres"` Postgres Postgres `toml:"postgres" comment:"Postgres configuration"`
} }
config := Config{Postgres{User: "pelletier", Password: "mypassword"}} config := Config{Postgres{User: "pelletier", Password: "mypassword", Database: "old_database"}}
b, err := toml.Marshal(config) b, err := toml.Marshal(config)
if err != nil { if err != nil {
log.Fatal(err) log.Fatal(err)
} }
fmt.Println(string(b)) fmt.Println(string(b))
// Output: // Output:
// # Postgres configuration
// [postgres] // [postgres]
//
// # not used anymore
// # db = "old_database"
// password = "mypassword" // password = "mypassword"
// user = "pelletier" // user = "pelletier"
} }
+31
View File
@@ -0,0 +1,31 @@
// +build gofuzz
package toml
func Fuzz(data []byte) int {
tree, err := LoadBytes(data)
if err != nil {
if tree != nil {
panic("tree must be nil if there is an error")
}
return 0
}
str, err := tree.ToTomlString()
if err != nil {
if str != "" {
panic(`str must be "" if there is an error`)
}
panic(err)
}
tree, err = Load(str)
if err != nil {
if tree != nil {
panic("tree must be nil if there is an error")
}
return 0
}
return 1
}
Executable
+15
View File
@@ -0,0 +1,15 @@
#! /bin/sh
set -eu
go get github.com/dvyukov/go-fuzz/go-fuzz
go get github.com/dvyukov/go-fuzz/go-fuzz-build
if [ ! -e toml-fuzz.zip ]; then
go-fuzz-build github.com/pelletier/go-toml
fi
rm -fr fuzz
mkdir -p fuzz/corpus
cp *.toml fuzz/corpus
go-fuzz -bin=toml-fuzz.zip -workdir=fuzz
+3 -12
View File
@@ -9,12 +9,14 @@ import (
"unicode" "unicode"
) )
// Convert the bare key group string to an array.
// The input supports double quotation to allow "." inside the key name,
// but escape sequences are not supported. Lexers must unescape them beforehand.
func parseKey(key string) ([]string, error) { func parseKey(key string) ([]string, error) {
groups := []string{} groups := []string{}
var buffer bytes.Buffer var buffer bytes.Buffer
inQuotes := false inQuotes := false
wasInQuotes := false wasInQuotes := false
escapeNext := false
ignoreSpace := true ignoreSpace := true
expectDot := false expectDot := false
@@ -25,15 +27,7 @@ func parseKey(key string) ([]string, error) {
} }
ignoreSpace = false ignoreSpace = false
} }
if escapeNext {
buffer.WriteRune(char)
escapeNext = false
continue
}
switch char { switch char {
case '\\':
escapeNext = true
continue
case '"': case '"':
if inQuotes { if inQuotes {
groups = append(groups, buffer.String()) groups = append(groups, buffer.String())
@@ -77,9 +71,6 @@ func parseKey(key string) ([]string, error) {
if inQuotes { if inQuotes {
return nil, errors.New("mismatched quotes") return nil, errors.New("mismatched quotes")
} }
if escapeNext {
return nil, errors.New("unfinished escape sequence")
}
if buffer.Len() > 0 { if buffer.Len() > 0 {
groups = append(groups, buffer.String()) groups = append(groups, buffer.String())
} }
+8 -1
View File
@@ -22,7 +22,10 @@ func testResult(t *testing.T, key string, expected []string) {
} }
func testError(t *testing.T, key string, expectedError string) { func testError(t *testing.T, key string, expectedError string) {
_, err := parseKey(key) res, err := parseKey(key)
if err == nil {
t.Fatalf("Expected error, but succesfully parsed key %s", res)
}
if fmt.Sprintf("%s", err) != expectedError { if fmt.Sprintf("%s", err) != expectedError {
t.Fatalf("Expected error \"%s\", but got \"%s\".", expectedError, err) t.Fatalf("Expected error \"%s\", but got \"%s\".", expectedError, err)
} }
@@ -47,6 +50,10 @@ func TestBaseKeyPound(t *testing.T) {
func TestQuotedKeys(t *testing.T) { func TestQuotedKeys(t *testing.T) {
testResult(t, `hello."foo".bar`, []string{"hello", "foo", "bar"}) testResult(t, `hello."foo".bar`, []string{"hello", "foo", "bar"})
testResult(t, `"hello!"`, []string{"hello!"}) testResult(t, `"hello!"`, []string{"hello!"})
testResult(t, `foo."ba.r".baz`, []string{"foo", "ba.r", "baz"})
// escape sequences must not be converted
testResult(t, `"hello\tworld"`, []string{`hello\tworld`})
} }
func TestEmptyKey(t *testing.T) { func TestEmptyKey(t *testing.T) {
+100 -1
View File
@@ -204,6 +204,14 @@ func (l *tomlLexer) lexRvalue() tomlLexStateFn {
return l.lexFalse return l.lexFalse
} }
if l.follow("inf") {
return l.lexInf
}
if l.follow("nan") {
return l.lexNan
}
if isSpace(next) { if isSpace(next) {
l.skip() l.skip()
continue continue
@@ -265,6 +273,18 @@ func (l *tomlLexer) lexFalse() tomlLexStateFn {
return l.lexRvalue return l.lexRvalue
} }
func (l *tomlLexer) lexInf() tomlLexStateFn {
l.fastForward(3)
l.emit(tokenInf)
return l.lexRvalue
}
func (l *tomlLexer) lexNan() tomlLexStateFn {
l.fastForward(3)
l.emit(tokenNan)
return l.lexRvalue
}
func (l *tomlLexer) lexEqual() tomlLexStateFn { func (l *tomlLexer) lexEqual() tomlLexStateFn {
l.next() l.next()
l.emit(tokenEqual) l.emit(tokenEqual)
@@ -277,6 +297,8 @@ func (l *tomlLexer) lexComma() tomlLexStateFn {
return l.lexRvalue return l.lexRvalue
} }
// Parse the key and emits its value without escape sequences.
// bare keys, basic string keys and literal string keys are supported.
func (l *tomlLexer) lexKey() tomlLexStateFn { func (l *tomlLexer) lexKey() tomlLexStateFn {
growingString := "" growingString := ""
@@ -287,7 +309,16 @@ func (l *tomlLexer) lexKey() tomlLexStateFn {
if err != nil { if err != nil {
return l.errorf(err.Error()) return l.errorf(err.Error())
} }
growingString += `"` + str + `"` growingString += str
l.next()
continue
} else if r == '\'' {
l.next()
str, err := l.lexLiteralStringAsString(`'`, false)
if err != nil {
return l.errorf(err.Error())
}
growingString += str
l.next() l.next()
continue continue
} else if r == '\n' { } else if r == '\n' {
@@ -527,6 +558,7 @@ func (l *tomlLexer) lexTableKey() tomlLexStateFn {
return l.lexInsideTableKey return l.lexInsideTableKey
} }
// Parse the key till "]]", but only bare keys are supported
func (l *tomlLexer) lexInsideTableArrayKey() tomlLexStateFn { func (l *tomlLexer) lexInsideTableArrayKey() tomlLexStateFn {
for r := l.peek(); r != eof; r = l.peek() { for r := l.peek(); r != eof; r = l.peek() {
switch r { switch r {
@@ -550,6 +582,7 @@ func (l *tomlLexer) lexInsideTableArrayKey() tomlLexStateFn {
return l.errorf("unclosed table array key") return l.errorf("unclosed table array key")
} }
// Parse the key till "]" but only bare keys are supported
func (l *tomlLexer) lexInsideTableKey() tomlLexStateFn { func (l *tomlLexer) lexInsideTableKey() tomlLexStateFn {
for r := l.peek(); r != eof; r = l.peek() { for r := l.peek(); r != eof; r = l.peek() {
switch r { switch r {
@@ -575,11 +608,77 @@ func (l *tomlLexer) lexRightBracket() tomlLexStateFn {
return l.lexRvalue return l.lexRvalue
} }
type validRuneFn func(r rune) bool
func isValidHexRune(r rune) bool {
return r >= 'a' && r <= 'f' ||
r >= 'A' && r <= 'F' ||
r >= '0' && r <= '9' ||
r == '_'
}
func isValidOctalRune(r rune) bool {
return r >= '0' && r <= '7' || r == '_'
}
func isValidBinaryRune(r rune) bool {
return r == '0' || r == '1' || r == '_'
}
func (l *tomlLexer) lexNumber() tomlLexStateFn { func (l *tomlLexer) lexNumber() tomlLexStateFn {
r := l.peek() r := l.peek()
if r == '0' {
follow := l.peekString(2)
if len(follow) == 2 {
var isValidRune validRuneFn
switch follow[1] {
case 'x':
isValidRune = isValidHexRune
case 'o':
isValidRune = isValidOctalRune
case 'b':
isValidRune = isValidBinaryRune
default:
if follow[1] >= 'a' && follow[1] <= 'z' || follow[1] >= 'A' && follow[1] <= 'Z' {
return l.errorf("unknown number base: %s. possible options are x (hex) o (octal) b (binary)", string(follow[1]))
}
}
if isValidRune != nil {
l.next()
l.next()
digitSeen := false
for {
next := l.peek()
if !isValidRune(next) {
break
}
digitSeen = true
l.next()
}
if !digitSeen {
return l.errorf("number needs at least one digit")
}
l.emit(tokenInteger)
return l.lexRvalue
}
}
}
if r == '+' || r == '-' { if r == '+' || r == '-' {
l.next() l.next()
if l.follow("inf") {
return l.lexInf
}
if l.follow("nan") {
return l.lexNan
}
} }
pointSeen := false pointSeen := false
expSeen := false expSeen := false
digitSeen := false digitSeen := false
+1 -1
View File
@@ -690,7 +690,7 @@ func TestKeyGroupArray(t *testing.T) {
func TestQuotedKey(t *testing.T) { func TestQuotedKey(t *testing.T) {
testFlow(t, "\"a b\" = 42", []token{ testFlow(t, "\"a b\" = 42", []token{
{Position{1, 1}, tokenKey, "\"a b\""}, {Position{1, 1}, tokenKey, "a b"},
{Position{1, 7}, tokenEqual, "="}, {Position{1, 7}, tokenEqual, "="},
{Position{1, 9}, tokenInteger, "42"}, {Position{1, 9}, tokenInteger, "42"},
{Position{1, 11}, tokenEOF, ""}, {Position{1, 11}, tokenEOF, ""},
+257 -137
View File
@@ -4,17 +4,33 @@ import (
"bytes" "bytes"
"errors" "errors"
"fmt" "fmt"
"io"
"reflect" "reflect"
"strconv"
"strings" "strings"
"time" "time"
) )
const tagKeyMultiline = "multiline"
type tomlOpts struct { type tomlOpts struct {
name string name string
comment string
commented bool
multiline bool
include bool include bool
omitempty bool omitempty bool
} }
type encOpts struct {
quoteMapKeys bool
arraysOneElementPerLine bool
}
var encOptsDefaults = encOpts{
quoteMapKeys: false,
}
var timeType = reflect.TypeOf(time.Time{}) var timeType = reflect.TypeOf(time.Time{})
var marshalerType = reflect.TypeOf(new(Marshaler)).Elem() var marshalerType = reflect.TypeOf(new(Marshaler)).Elem()
@@ -94,8 +110,15 @@ encoder, except that there is no concept of a Marshaler interface or MarshalTOML
function for sub-structs, and currently only definite types can be marshaled function for sub-structs, and currently only definite types can be marshaled
(i.e. no `interface{}`). (i.e. no `interface{}`).
The following struct annotations are supported:
toml:"Field" Overrides the field's name to output.
omitempty When set, empty values and groups are not emitted.
comment:"comment" Emits a # comment on the same line. This supports new lines.
commented:"true" Emits the value as commented.
Note that pointers are automatically assigned the "omitempty" option, as TOML Note that pointers are automatically assigned the "omitempty" option, as TOML
explicity does not handle null values (saying instead the label should be explicitly does not handle null values (saying instead the label should be
dropped). dropped).
Tree structural types and corresponding marshal types: Tree structural types and corresponding marshal types:
@@ -115,6 +138,66 @@ Tree primitive types and corresponding marshal types:
time.Time time.Time{}, pointers to same time.Time time.Time{}, pointers to same
*/ */
func Marshal(v interface{}) ([]byte, error) { func Marshal(v interface{}) ([]byte, error) {
return NewEncoder(nil).marshal(v)
}
// Encoder writes TOML values to an output stream.
type Encoder struct {
w io.Writer
encOpts
}
// NewEncoder returns a new encoder that writes to w.
func NewEncoder(w io.Writer) *Encoder {
return &Encoder{
w: w,
encOpts: encOptsDefaults,
}
}
// Encode writes the TOML encoding of v to the stream.
//
// See the documentation for Marshal for details.
func (e *Encoder) Encode(v interface{}) error {
b, err := e.marshal(v)
if err != nil {
return err
}
if _, err := e.w.Write(b); err != nil {
return err
}
return nil
}
// QuoteMapKeys sets up the encoder to encode
// maps with string type keys with quoted TOML keys.
//
// This relieves the character limitations on map keys.
func (e *Encoder) QuoteMapKeys(v bool) *Encoder {
e.quoteMapKeys = v
return e
}
// ArraysWithOneElementPerLine sets up the encoder to encode arrays
// with more than one element on multiple lines instead of one.
//
// For example:
//
// A = [1,2,3]
//
// Becomes
//
// A = [
// 1,
// 2,
// 3,
// ]
func (e *Encoder) ArraysWithOneElementPerLine(v bool) *Encoder {
e.arraysOneElementPerLine = v
return e
}
func (e *Encoder) marshal(v interface{}) ([]byte, error) {
mtype := reflect.TypeOf(v) mtype := reflect.TypeOf(v)
if mtype.Kind() != reflect.Struct { if mtype.Kind() != reflect.Struct {
return []byte{}, errors.New("Only a struct can be marshaled to TOML") return []byte{}, errors.New("Only a struct can be marshaled to TOML")
@@ -123,18 +206,21 @@ func Marshal(v interface{}) ([]byte, error) {
if isCustomMarshaler(mtype) { if isCustomMarshaler(mtype) {
return callCustomMarshaler(sval) return callCustomMarshaler(sval)
} }
t, err := valueToTree(mtype, sval) t, err := e.valueToTree(mtype, sval)
if err != nil { if err != nil {
return []byte{}, err return []byte{}, err
} }
s, err := t.ToTomlString()
return []byte(s), err var buf bytes.Buffer
_, err = t.writeTo(&buf, "", "", 0, e.arraysOneElementPerLine)
return buf.Bytes(), err
} }
// Convert given marshal struct or map value to toml tree // Convert given marshal struct or map value to toml tree
func valueToTree(mtype reflect.Type, mval reflect.Value) (*Tree, error) { func (e *Encoder) valueToTree(mtype reflect.Type, mval reflect.Value) (*Tree, error) {
if mtype.Kind() == reflect.Ptr { if mtype.Kind() == reflect.Ptr {
return valueToTree(mtype.Elem(), mval.Elem()) return e.valueToTree(mtype.Elem(), mval.Elem())
} }
tval := newTree() tval := newTree()
switch mtype.Kind() { switch mtype.Kind() {
@@ -143,31 +229,44 @@ func valueToTree(mtype reflect.Type, mval reflect.Value) (*Tree, error) {
mtypef, mvalf := mtype.Field(i), mval.Field(i) mtypef, mvalf := mtype.Field(i), mval.Field(i)
opts := tomlOptions(mtypef) opts := tomlOptions(mtypef)
if opts.include && (!opts.omitempty || !isZero(mvalf)) { if opts.include && (!opts.omitempty || !isZero(mvalf)) {
val, err := valueToToml(mtypef.Type, mvalf) val, err := e.valueToToml(mtypef.Type, mvalf)
if err != nil { if err != nil {
return nil, err return nil, err
} }
tval.Set(opts.name, val)
tval.SetWithOptions(opts.name, SetOptions{
Comment: opts.comment,
Commented: opts.commented,
Multiline: opts.multiline,
}, val)
} }
} }
case reflect.Map: case reflect.Map:
for _, key := range mval.MapKeys() { for _, key := range mval.MapKeys() {
mvalf := mval.MapIndex(key) mvalf := mval.MapIndex(key)
val, err := valueToToml(mtype.Elem(), mvalf) val, err := e.valueToToml(mtype.Elem(), mvalf)
if err != nil { if err != nil {
return nil, err return nil, err
} }
tval.Set(key.String(), val) if e.quoteMapKeys {
keyStr, err := tomlValueStringRepresentation(key.String(), "", e.arraysOneElementPerLine)
if err != nil {
return nil, err
}
tval.SetPath([]string{keyStr}, val)
} else {
tval.Set(key.String(), val)
}
} }
} }
return tval, nil return tval, nil
} }
// Convert given marshal slice to slice of Toml trees // Convert given marshal slice to slice of Toml trees
func valueToTreeSlice(mtype reflect.Type, mval reflect.Value) ([]*Tree, error) { func (e *Encoder) valueToTreeSlice(mtype reflect.Type, mval reflect.Value) ([]*Tree, error) {
tval := make([]*Tree, mval.Len(), mval.Len()) tval := make([]*Tree, mval.Len(), mval.Len())
for i := 0; i < mval.Len(); i++ { for i := 0; i < mval.Len(); i++ {
val, err := valueToTree(mtype.Elem(), mval.Index(i)) val, err := e.valueToTree(mtype.Elem(), mval.Index(i))
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -177,10 +276,10 @@ func valueToTreeSlice(mtype reflect.Type, mval reflect.Value) ([]*Tree, error) {
} }
// Convert given marshal slice to slice of toml values // Convert given marshal slice to slice of toml values
func valueToOtherSlice(mtype reflect.Type, mval reflect.Value) (interface{}, error) { func (e *Encoder) valueToOtherSlice(mtype reflect.Type, mval reflect.Value) (interface{}, error) {
tval := make([]interface{}, mval.Len(), mval.Len()) tval := make([]interface{}, mval.Len(), mval.Len())
for i := 0; i < mval.Len(); i++ { for i := 0; i < mval.Len(); i++ {
val, err := valueToToml(mtype.Elem(), mval.Index(i)) val, err := e.valueToToml(mtype.Elem(), mval.Index(i))
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -190,19 +289,19 @@ func valueToOtherSlice(mtype reflect.Type, mval reflect.Value) (interface{}, err
} }
// Convert given marshal value to toml value // Convert given marshal value to toml value
func valueToToml(mtype reflect.Type, mval reflect.Value) (interface{}, error) { func (e *Encoder) valueToToml(mtype reflect.Type, mval reflect.Value) (interface{}, error) {
if mtype.Kind() == reflect.Ptr { if mtype.Kind() == reflect.Ptr {
return valueToToml(mtype.Elem(), mval.Elem()) return e.valueToToml(mtype.Elem(), mval.Elem())
} }
switch { switch {
case isCustomMarshaler(mtype): case isCustomMarshaler(mtype):
return callCustomMarshaler(mval) return callCustomMarshaler(mval)
case isTree(mtype): case isTree(mtype):
return valueToTree(mtype, mval) return e.valueToTree(mtype, mval)
case isTreeSlice(mtype): case isTreeSlice(mtype):
return valueToTreeSlice(mtype, mval) return e.valueToTreeSlice(mtype, mval)
case isOtherSlice(mtype): case isOtherSlice(mtype):
return valueToOtherSlice(mtype, mval) return e.valueToOtherSlice(mtype, mval)
default: default:
switch mtype.Kind() { switch mtype.Kind() {
case reflect.Bool: case reflect.Bool:
@@ -227,17 +326,16 @@ func valueToToml(mtype reflect.Type, mval reflect.Value) (interface{}, error) {
// Neither Unmarshaler interfaces nor UnmarshalTOML functions are supported for // Neither Unmarshaler interfaces nor UnmarshalTOML functions are supported for
// sub-structs, and only definite types can be unmarshaled. // sub-structs, and only definite types can be unmarshaled.
func (t *Tree) Unmarshal(v interface{}) error { func (t *Tree) Unmarshal(v interface{}) error {
mtype := reflect.TypeOf(v) d := Decoder{tval: t}
if mtype.Kind() != reflect.Ptr || mtype.Elem().Kind() != reflect.Struct { return d.unmarshal(v)
return errors.New("Only a pointer to struct can be unmarshaled from TOML") }
}
sval, err := valueFromTree(mtype.Elem(), t) // Marshal returns the TOML encoding of Tree.
if err != nil { // See Marshal() documentation for types mapping table.
return err func (t *Tree) Marshal() ([]byte, error) {
} var buf bytes.Buffer
reflect.ValueOf(v).Elem().Set(sval) err := NewEncoder(&buf).Encode(t)
return nil return buf.Bytes(), err
} }
// Unmarshal parses the TOML-encoded data and stores the result in the value // Unmarshal parses the TOML-encoded data and stores the result in the value
@@ -246,6 +344,10 @@ func (t *Tree) Unmarshal(v interface{}) error {
// sub-structs, and currently only definite types can be unmarshaled to (i.e. no // sub-structs, and currently only definite types can be unmarshaled to (i.e. no
// `interface{}`). // `interface{}`).
// //
// The following struct annotations are supported:
//
// toml:"Field" Overrides the field's name to map to.
//
// See Marshal() documentation for types mapping table. // See Marshal() documentation for types mapping table.
func Unmarshal(data []byte, v interface{}) error { func Unmarshal(data []byte, v interface{}) error {
t, err := LoadReader(bytes.NewReader(data)) t, err := LoadReader(bytes.NewReader(data))
@@ -255,10 +357,52 @@ func Unmarshal(data []byte, v interface{}) error {
return t.Unmarshal(v) return t.Unmarshal(v)
} }
// Decoder reads and decodes TOML values from an input stream.
type Decoder struct {
r io.Reader
tval *Tree
encOpts
}
// NewDecoder returns a new decoder that reads from r.
func NewDecoder(r io.Reader) *Decoder {
return &Decoder{
r: r,
encOpts: encOptsDefaults,
}
}
// Decode reads a TOML-encoded value from it's input
// and unmarshals it in the value pointed at by v.
//
// See the documentation for Marshal for details.
func (d *Decoder) Decode(v interface{}) error {
var err error
d.tval, err = LoadReader(d.r)
if err != nil {
return err
}
return d.unmarshal(v)
}
func (d *Decoder) unmarshal(v interface{}) error {
mtype := reflect.TypeOf(v)
if mtype.Kind() != reflect.Ptr || mtype.Elem().Kind() != reflect.Struct {
return errors.New("Only a pointer to struct can be unmarshaled from TOML")
}
sval, err := d.valueFromTree(mtype.Elem(), d.tval)
if err != nil {
return err
}
reflect.ValueOf(v).Elem().Set(sval)
return nil
}
// Convert toml tree to marshal struct or map, using marshal type // Convert toml tree to marshal struct or map, using marshal type
func valueFromTree(mtype reflect.Type, tval *Tree) (reflect.Value, error) { func (d *Decoder) valueFromTree(mtype reflect.Type, tval *Tree) (reflect.Value, error) {
if mtype.Kind() == reflect.Ptr { if mtype.Kind() == reflect.Ptr {
return unwrapPointer(mtype, tval) return d.unwrapPointer(mtype, tval)
} }
var mval reflect.Value var mval reflect.Value
switch mtype.Kind() { switch mtype.Kind() {
@@ -276,7 +420,7 @@ func valueFromTree(mtype reflect.Type, tval *Tree) (reflect.Value, error) {
continue continue
} }
val := tval.Get(key) val := tval.Get(key)
mvalf, err := valueFromToml(mtypef.Type, val) mvalf, err := d.valueFromToml(mtypef.Type, val)
if err != nil { if err != nil {
return mval, formatError(err, tval.GetPosition(key)) return mval, formatError(err, tval.GetPosition(key))
} }
@@ -288,8 +432,9 @@ func valueFromTree(mtype reflect.Type, tval *Tree) (reflect.Value, error) {
case reflect.Map: case reflect.Map:
mval = reflect.MakeMap(mtype) mval = reflect.MakeMap(mtype)
for _, key := range tval.Keys() { for _, key := range tval.Keys() {
val := tval.Get(key) // TODO: path splits key
mvalf, err := valueFromToml(mtype.Elem(), val) val := tval.GetPath([]string{key})
mvalf, err := d.valueFromToml(mtype.Elem(), val)
if err != nil { if err != nil {
return mval, formatError(err, tval.GetPosition(key)) return mval, formatError(err, tval.GetPosition(key))
} }
@@ -300,10 +445,10 @@ func valueFromTree(mtype reflect.Type, tval *Tree) (reflect.Value, error) {
} }
// Convert toml value to marshal struct/map slice, using marshal type // Convert toml value to marshal struct/map slice, using marshal type
func valueFromTreeSlice(mtype reflect.Type, tval []*Tree) (reflect.Value, error) { func (d *Decoder) valueFromTreeSlice(mtype reflect.Type, tval []*Tree) (reflect.Value, error) {
mval := reflect.MakeSlice(mtype, len(tval), len(tval)) mval := reflect.MakeSlice(mtype, len(tval), len(tval))
for i := 0; i < len(tval); i++ { for i := 0; i < len(tval); i++ {
val, err := valueFromTree(mtype.Elem(), tval[i]) val, err := d.valueFromTree(mtype.Elem(), tval[i])
if err != nil { if err != nil {
return mval, err return mval, err
} }
@@ -313,10 +458,10 @@ func valueFromTreeSlice(mtype reflect.Type, tval []*Tree) (reflect.Value, error)
} }
// Convert toml value to marshal primitive slice, using marshal type // Convert toml value to marshal primitive slice, using marshal type
func valueFromOtherSlice(mtype reflect.Type, tval []interface{}) (reflect.Value, error) { func (d *Decoder) valueFromOtherSlice(mtype reflect.Type, tval []interface{}) (reflect.Value, error) {
mval := reflect.MakeSlice(mtype, len(tval), len(tval)) mval := reflect.MakeSlice(mtype, len(tval), len(tval))
for i := 0; i < len(tval); i++ { for i := 0; i < len(tval); i++ {
val, err := valueFromToml(mtype.Elem(), tval[i]) val, err := d.valueFromToml(mtype.Elem(), tval[i])
if err != nil { if err != nil {
return mval, err return mval, err
} }
@@ -326,117 +471,86 @@ func valueFromOtherSlice(mtype reflect.Type, tval []interface{}) (reflect.Value,
} }
// Convert toml value to marshal value, using marshal type // Convert toml value to marshal value, using marshal type
func valueFromToml(mtype reflect.Type, tval interface{}) (reflect.Value, error) { func (d *Decoder) valueFromToml(mtype reflect.Type, tval interface{}) (reflect.Value, error) {
if mtype.Kind() == reflect.Ptr { if mtype.Kind() == reflect.Ptr {
return unwrapPointer(mtype, tval) return d.unwrapPointer(mtype, tval)
} }
switch {
case isTree(mtype): switch tval.(type) {
return valueFromTree(mtype, tval.(*Tree)) case *Tree:
case isTreeSlice(mtype): if isTree(mtype) {
return valueFromTreeSlice(mtype, tval.([]*Tree)) return d.valueFromTree(mtype, tval.(*Tree))
case isOtherSlice(mtype): }
return valueFromOtherSlice(mtype, tval.([]interface{})) return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to a tree", tval, tval)
case []*Tree:
if isTreeSlice(mtype) {
return d.valueFromTreeSlice(mtype, tval.([]*Tree))
}
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to trees", tval, tval)
case []interface{}:
if isOtherSlice(mtype) {
return d.valueFromOtherSlice(mtype, tval.([]interface{}))
}
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to a slice", tval, tval)
default: default:
switch mtype.Kind() { switch mtype.Kind() {
case reflect.Bool: case reflect.Bool, reflect.Struct:
val, ok := tval.(bool) val := reflect.ValueOf(tval)
if !ok { // if this passes for when mtype is reflect.Struct, tval is a time.Time
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to bool", tval, tval) if !val.Type().ConvertibleTo(mtype) {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to %v", tval, tval, mtype.String())
} }
return reflect.ValueOf(val), nil
case reflect.Int: return val.Convert(mtype), nil
val, ok := tval.(int64)
if !ok {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to int", tval, tval)
}
return reflect.ValueOf(int(val)), nil
case reflect.Int8:
val, ok := tval.(int64)
if !ok {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to int", tval, tval)
}
return reflect.ValueOf(int8(val)), nil
case reflect.Int16:
val, ok := tval.(int64)
if !ok {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to int", tval, tval)
}
return reflect.ValueOf(int16(val)), nil
case reflect.Int32:
val, ok := tval.(int64)
if !ok {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to int", tval, tval)
}
return reflect.ValueOf(int32(val)), nil
case reflect.Int64:
val, ok := tval.(int64)
if !ok {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to int", tval, tval)
}
return reflect.ValueOf(val), nil
case reflect.Uint:
val, ok := tval.(int64)
if !ok {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to uint", tval, tval)
}
return reflect.ValueOf(uint(val)), nil
case reflect.Uint8:
val, ok := tval.(int64)
if !ok {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to uint", tval, tval)
}
return reflect.ValueOf(uint8(val)), nil
case reflect.Uint16:
val, ok := tval.(int64)
if !ok {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to uint", tval, tval)
}
return reflect.ValueOf(uint16(val)), nil
case reflect.Uint32:
val, ok := tval.(int64)
if !ok {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to uint", tval, tval)
}
return reflect.ValueOf(uint32(val)), nil
case reflect.Uint64:
val, ok := tval.(int64)
if !ok {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to uint", tval, tval)
}
return reflect.ValueOf(uint64(val)), nil
case reflect.Float32:
val, ok := tval.(float64)
if !ok {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to float", tval, tval)
}
return reflect.ValueOf(float32(val)), nil
case reflect.Float64:
val, ok := tval.(float64)
if !ok {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to float", tval, tval)
}
return reflect.ValueOf(val), nil
case reflect.String: case reflect.String:
val, ok := tval.(string) val := reflect.ValueOf(tval)
if !ok { // stupidly, int64 is convertible to string. So special case this.
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to string", tval, tval) if !val.Type().ConvertibleTo(mtype) || val.Kind() == reflect.Int64 {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to %v", tval, tval, mtype.String())
} }
return reflect.ValueOf(val), nil
case reflect.Struct: return val.Convert(mtype), nil
val, ok := tval.(time.Time) case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
if !ok { val := reflect.ValueOf(tval)
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to time", tval, tval) if !val.Type().ConvertibleTo(mtype) {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to %v", tval, tval, mtype.String())
} }
return reflect.ValueOf(val), nil if reflect.Indirect(reflect.New(mtype)).OverflowInt(val.Int()) {
return reflect.ValueOf(nil), fmt.Errorf("%v(%T) would overflow %v", tval, tval, mtype.String())
}
return val.Convert(mtype), nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
val := reflect.ValueOf(tval)
if !val.Type().ConvertibleTo(mtype) {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to %v", tval, tval, mtype.String())
}
if val.Int() < 0 {
return reflect.ValueOf(nil), fmt.Errorf("%v(%T) is negative so does not fit in %v", tval, tval, mtype.String())
}
if reflect.Indirect(reflect.New(mtype)).OverflowUint(uint64(val.Int())) {
return reflect.ValueOf(nil), fmt.Errorf("%v(%T) would overflow %v", tval, tval, mtype.String())
}
return val.Convert(mtype), nil
case reflect.Float32, reflect.Float64:
val := reflect.ValueOf(tval)
if !val.Type().ConvertibleTo(mtype) {
return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to %v", tval, tval, mtype.String())
}
if reflect.Indirect(reflect.New(mtype)).OverflowFloat(val.Float()) {
return reflect.ValueOf(nil), fmt.Errorf("%v(%T) would overflow %v", tval, tval, mtype.String())
}
return val.Convert(mtype), nil
default: default:
return reflect.ValueOf(nil), fmt.Errorf("Unmarshal can't handle %v(%v)", mtype, mtype.Kind()) return reflect.ValueOf(nil), fmt.Errorf("Can't convert %v(%T) to %v(%v)", tval, tval, mtype, mtype.Kind())
} }
} }
} }
func unwrapPointer(mtype reflect.Type, tval interface{}) (reflect.Value, error) { func (d *Decoder) unwrapPointer(mtype reflect.Type, tval interface{}) (reflect.Value, error) {
val, err := valueFromToml(mtype.Elem(), tval) val, err := d.valueFromToml(mtype.Elem(), tval)
if err != nil { if err != nil {
return reflect.ValueOf(nil), err return reflect.ValueOf(nil), err
} }
@@ -448,7 +562,13 @@ func unwrapPointer(mtype reflect.Type, tval interface{}) (reflect.Value, error)
func tomlOptions(vf reflect.StructField) tomlOpts { func tomlOptions(vf reflect.StructField) tomlOpts {
tag := vf.Tag.Get("toml") tag := vf.Tag.Get("toml")
parse := strings.Split(tag, ",") parse := strings.Split(tag, ",")
result := tomlOpts{vf.Name, true, false} var comment string
if c := vf.Tag.Get("comment"); c != "" {
comment = c
}
commented, _ := strconv.ParseBool(vf.Tag.Get("commented"))
multiline, _ := strconv.ParseBool(vf.Tag.Get(tagKeyMultiline))
result := tomlOpts{name: vf.Name, comment: comment, commented: commented, multiline: multiline, include: true, omitempty: false}
if parse[0] != "" { if parse[0] != "" {
if parse[0] == "-" && len(parse) == 1 { if parse[0] == "-" && len(parse) == 1 {
result.include = false result.include = false
+209 -3
View File
@@ -6,6 +6,7 @@ import (
"fmt" "fmt"
"io/ioutil" "io/ioutil"
"reflect" "reflect"
"strings"
"testing" "testing"
"time" "time"
) )
@@ -145,8 +146,8 @@ var docData = testDoc{
Second: &subdoc, Second: &subdoc,
}, },
SubDocList: []testSubDoc{ SubDocList: []testSubDoc{
testSubDoc{"List.First", 0}, {"List.First", 0},
testSubDoc{"List.Second", 0}, {"List.Second", 0},
}, },
SubDocPtrs: []*testSubDoc{&subdoc}, SubDocPtrs: []*testSubDoc{&subdoc},
} }
@@ -508,6 +509,14 @@ func TestPointerUnmarshal(t *testing.T) {
} }
} }
func TestUnmarshalTypeMismatch(t *testing.T) {
result := pointerMarshalTestStruct{}
err := Unmarshal([]byte("List = 123"), &result)
if !strings.HasPrefix(err.Error(), "(1, 1): Can't convert 123(int64) to []string(slice)") {
t.Errorf("Type mismatch must be reported: got %v", err.Error())
}
}
type nestedMarshalTestStruct struct { type nestedMarshalTestStruct struct {
String [][]string String [][]string
//Struct [][]basicMarshalTestSubStruct //Struct [][]basicMarshalTestSubStruct
@@ -521,7 +530,7 @@ var strPtr = []*string{&str1, &str2}
var strPtr2 = []*[]*string{&strPtr} var strPtr2 = []*[]*string{&strPtr}
var nestedTestData = nestedMarshalTestStruct{ var nestedTestData = nestedMarshalTestStruct{
String: [][]string{[]string{"Five", "Six"}, []string{"One", "Two"}}, String: [][]string{{"Five", "Six"}, {"One", "Two"}},
StringPtr: &strPtr2, StringPtr: &strPtr2,
} }
@@ -598,3 +607,200 @@ func TestNestedCustomMarshaler(t *testing.T) {
t.Errorf("Bad nested custom marshaler: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", expected, result) t.Errorf("Bad nested custom marshaler: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", expected, result)
} }
} }
var commentTestToml = []byte(`
# it's a comment on type
[postgres]
# isCommented = "dvalue"
noComment = "cvalue"
# A comment on AttrB with a
# break line
password = "bvalue"
# A comment on AttrA
user = "avalue"
[[postgres.My]]
# a comment on my on typeC
My = "Foo"
[[postgres.My]]
# a comment on my on typeC
My = "Baar"
`)
func TestMarshalComment(t *testing.T) {
type TypeC struct {
My string `comment:"a comment on my on typeC"`
}
type TypeB struct {
AttrA string `toml:"user" comment:"A comment on AttrA"`
AttrB string `toml:"password" comment:"A comment on AttrB with a\n break line"`
AttrC string `toml:"noComment"`
AttrD string `toml:"isCommented" commented:"true"`
My []TypeC
}
type TypeA struct {
TypeB TypeB `toml:"postgres" comment:"it's a comment on type"`
}
ta := []TypeC{{My: "Foo"}, {My: "Baar"}}
config := TypeA{TypeB{AttrA: "avalue", AttrB: "bvalue", AttrC: "cvalue", AttrD: "dvalue", My: ta}}
result, err := Marshal(config)
if err != nil {
t.Fatal(err)
}
expected := commentTestToml
if !bytes.Equal(result, expected) {
t.Errorf("Bad marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", expected, result)
}
}
type mapsTestStruct struct {
Simple map[string]string
Paths map[string]string
Other map[string]float64
X struct {
Y struct {
Z map[string]bool
}
}
}
var mapsTestData = mapsTestStruct{
Simple: map[string]string{
"one plus one": "two",
"next": "three",
},
Paths: map[string]string{
"/this/is/a/path": "/this/is/also/a/path",
"/heloo.txt": "/tmp/lololo.txt",
},
Other: map[string]float64{
"testing": 3.9999,
},
X: struct{ Y struct{ Z map[string]bool } }{
Y: struct{ Z map[string]bool }{
Z: map[string]bool{
"is.Nested": true,
},
},
},
}
var mapsTestToml = []byte(`
[Other]
"testing" = 3.9999
[Paths]
"/heloo.txt" = "/tmp/lololo.txt"
"/this/is/a/path" = "/this/is/also/a/path"
[Simple]
"next" = "three"
"one plus one" = "two"
[X]
[X.Y]
[X.Y.Z]
"is.Nested" = true
`)
func TestEncodeQuotedMapKeys(t *testing.T) {
var buf bytes.Buffer
if err := NewEncoder(&buf).QuoteMapKeys(true).Encode(mapsTestData); err != nil {
t.Fatal(err)
}
result := buf.Bytes()
expected := mapsTestToml
if !bytes.Equal(result, expected) {
t.Errorf("Bad maps marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", expected, result)
}
}
func TestDecodeQuotedMapKeys(t *testing.T) {
result := mapsTestStruct{}
err := NewDecoder(bytes.NewBuffer(mapsTestToml)).Decode(&result)
expected := mapsTestData
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(result, expected) {
t.Errorf("Bad maps unmarshal: expected %v, got %v", expected, result)
}
}
type structArrayNoTag struct {
A struct {
B []int64
C []int64
}
}
func TestMarshalArray(t *testing.T) {
expected := []byte(`
[A]
B = [1,2,3]
C = [1]
`)
m := structArrayNoTag{
A: struct {
B []int64
C []int64
}{
B: []int64{1, 2, 3},
C: []int64{1},
},
}
b, err := Marshal(m)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(b, expected) {
t.Errorf("Bad arrays marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", expected, b)
}
}
func TestMarshalArrayOnePerLine(t *testing.T) {
expected := []byte(`
[A]
B = [
1,
2,
3,
]
C = [1]
`)
m := structArrayNoTag{
A: struct {
B []int64
C []int64
}{
B: []int64{1, 2, 3},
C: []int64{1},
},
}
var buf bytes.Buffer
encoder := NewEncoder(&buf).ArraysWithOneElementPerLine(true)
err := encoder.Encode(m)
if err != nil {
t.Fatal(err)
}
b := buf.Bytes()
if !bytes.Equal(b, expected) {
t.Errorf("Bad arrays marshal: expected\n-----\n%s\n-----\ngot\n-----\n%s\n-----\n", expected, b)
}
}
+63 -16
View File
@@ -5,6 +5,7 @@ package toml
import ( import (
"errors" "errors"
"fmt" "fmt"
"math"
"reflect" "reflect"
"regexp" "regexp"
"strconv" "strconv"
@@ -185,10 +186,7 @@ func (p *tomlParser) parseAssign() tomlParserStateFn {
} }
// assign value to the found table // assign value to the found table
keyVals, err := parseKey(key.val) keyVals := []string{key.val}
if err != nil {
p.raiseError(key, "%s", err)
}
if len(keyVals) != 1 { if len(keyVals) != 1 {
p.raiseError(key, "Invalid key") p.raiseError(key, "Invalid key")
} }
@@ -205,20 +203,32 @@ func (p *tomlParser) parseAssign() tomlParserStateFn {
case *Tree, []*Tree: case *Tree, []*Tree:
toInsert = value toInsert = value
default: default:
toInsert = &tomlValue{value, key.Position} toInsert = &tomlValue{value: value, position: key.Position}
} }
targetNode.values[keyVal] = toInsert targetNode.values[keyVal] = toInsert
return p.parseStart return p.parseStart
} }
var numberUnderscoreInvalidRegexp *regexp.Regexp var numberUnderscoreInvalidRegexp *regexp.Regexp
var hexNumberUnderscoreInvalidRegexp *regexp.Regexp
func cleanupNumberToken(value string) (string, error) { func numberContainsInvalidUnderscore(value string) error {
if numberUnderscoreInvalidRegexp.MatchString(value) { if numberUnderscoreInvalidRegexp.MatchString(value) {
return "", errors.New("invalid use of _ in number") return errors.New("invalid use of _ in number")
} }
return nil
}
func hexNumberContainsInvalidUnderscore(value string) error {
if hexNumberUnderscoreInvalidRegexp.MatchString(value) {
return errors.New("invalid use of _ in hex number")
}
return nil
}
func cleanupNumberToken(value string) string {
cleanedVal := strings.Replace(value, "_", "", -1) cleanedVal := strings.Replace(value, "_", "", -1)
return cleanedVal, nil return cleanedVal
} }
func (p *tomlParser) parseRvalue() interface{} { func (p *tomlParser) parseRvalue() interface{} {
@@ -234,21 +244,57 @@ func (p *tomlParser) parseRvalue() interface{} {
return true return true
case tokenFalse: case tokenFalse:
return false return false
case tokenInteger: case tokenInf:
cleanedVal, err := cleanupNumberToken(tok.val) if tok.val[0] == '-' {
if err != nil { return math.Inf(-1)
p.raiseError(tok, "%s", err) }
return math.Inf(1)
case tokenNan:
return math.NaN()
case tokenInteger:
cleanedVal := cleanupNumberToken(tok.val)
var err error
var val int64
if len(cleanedVal) >= 3 && cleanedVal[0] == '0' {
switch cleanedVal[1] {
case 'x':
err = hexNumberContainsInvalidUnderscore(tok.val)
if err != nil {
p.raiseError(tok, "%s", err)
}
val, err = strconv.ParseInt(cleanedVal[2:], 16, 64)
case 'o':
err = numberContainsInvalidUnderscore(tok.val)
if err != nil {
p.raiseError(tok, "%s", err)
}
val, err = strconv.ParseInt(cleanedVal[2:], 8, 64)
case 'b':
err = numberContainsInvalidUnderscore(tok.val)
if err != nil {
p.raiseError(tok, "%s", err)
}
val, err = strconv.ParseInt(cleanedVal[2:], 2, 64)
default:
panic("invalid base") // the lexer should catch this first
}
} else {
err = numberContainsInvalidUnderscore(tok.val)
if err != nil {
p.raiseError(tok, "%s", err)
}
val, err = strconv.ParseInt(cleanedVal, 10, 64)
} }
val, err := strconv.ParseInt(cleanedVal, 10, 64)
if err != nil { if err != nil {
p.raiseError(tok, "%s", err) p.raiseError(tok, "%s", err)
} }
return val return val
case tokenFloat: case tokenFloat:
cleanedVal, err := cleanupNumberToken(tok.val) err := numberContainsInvalidUnderscore(tok.val)
if err != nil { if err != nil {
p.raiseError(tok, "%s", err) p.raiseError(tok, "%s", err)
} }
cleanedVal := cleanupNumberToken(tok.val)
val, err := strconv.ParseFloat(cleanedVal, 64) val, err := strconv.ParseFloat(cleanedVal, 64)
if err != nil { if err != nil {
p.raiseError(tok, "%s", err) p.raiseError(tok, "%s", err)
@@ -309,7 +355,7 @@ Loop:
} }
p.getToken() p.getToken()
default: default:
p.raiseError(follow, "unexpected token type in inline table: %s", follow.typ.String()) p.raiseError(follow, "unexpected token type in inline table: %s", follow.String())
} }
previous = follow previous = follow
} }
@@ -379,5 +425,6 @@ func parseToml(flow []token) *Tree {
} }
func init() { func init() {
numberUnderscoreInvalidRegexp = regexp.MustCompile(`([^\d]_|_[^\d]|_$|^_)`) numberUnderscoreInvalidRegexp = regexp.MustCompile(`([^\d]_|_[^\d])|_$|^_`)
hexNumberUnderscoreInvalidRegexp = regexp.MustCompile(`(^0x_)|([^\da-f]_|_[^\da-f])|_$|^_`)
} }
+116 -2
View File
@@ -2,6 +2,7 @@ package toml
import ( import (
"fmt" "fmt"
"math"
"reflect" "reflect"
"testing" "testing"
"time" "time"
@@ -72,6 +73,17 @@ func TestNumberInKey(t *testing.T) {
}) })
} }
func TestIncorrectKeyExtraSquareBracket(t *testing.T) {
_, err := Load(`[a]b]
zyx = 42`)
if err == nil {
t.Error("Error should have been returned.")
}
if err.Error() != "(1, 4): unexpected token" {
t.Error("Bad error message:", err.Error())
}
}
func TestSimpleNumbers(t *testing.T) { func TestSimpleNumbers(t *testing.T) {
tree, err := Load("a = +42\nb = -21\nc = +4.2\nd = -2.1") tree, err := Load("a = +42\nb = -21\nc = +4.2\nd = -2.1")
assertTree(t, tree, err, map[string]interface{}{ assertTree(t, tree, err, map[string]interface{}{
@@ -82,6 +94,78 @@ func TestSimpleNumbers(t *testing.T) {
}) })
} }
func TestSpecialFloats(t *testing.T) {
tree, err := Load(`
normalinf = inf
plusinf = +inf
minusinf = -inf
normalnan = nan
plusnan = +nan
minusnan = -nan
`)
assertTree(t, tree, err, map[string]interface{}{
"normalinf": math.Inf(1),
"plusinf": math.Inf(1),
"minusinf": math.Inf(-1),
"normalnan": math.NaN(),
"plusnan": math.NaN(),
"minusnan": math.NaN(),
})
}
func TestHexIntegers(t *testing.T) {
tree, err := Load(`a = 0xDEADBEEF`)
assertTree(t, tree, err, map[string]interface{}{"a": int64(3735928559)})
tree, err = Load(`a = 0xdeadbeef`)
assertTree(t, tree, err, map[string]interface{}{"a": int64(3735928559)})
tree, err = Load(`a = 0xdead_beef`)
assertTree(t, tree, err, map[string]interface{}{"a": int64(3735928559)})
_, err = Load(`a = 0x_1`)
if err.Error() != "(1, 5): invalid use of _ in hex number" {
t.Error("Bad error message:", err.Error())
}
}
func TestOctIntegers(t *testing.T) {
tree, err := Load(`a = 0o01234567`)
assertTree(t, tree, err, map[string]interface{}{"a": int64(342391)})
tree, err = Load(`a = 0o755`)
assertTree(t, tree, err, map[string]interface{}{"a": int64(493)})
_, err = Load(`a = 0o_1`)
if err.Error() != "(1, 5): invalid use of _ in number" {
t.Error("Bad error message:", err.Error())
}
}
func TestBinIntegers(t *testing.T) {
tree, err := Load(`a = 0b11010110`)
assertTree(t, tree, err, map[string]interface{}{"a": int64(214)})
_, err = Load(`a = 0b_1`)
if err.Error() != "(1, 5): invalid use of _ in number" {
t.Error("Bad error message:", err.Error())
}
}
func TestBadIntegerBase(t *testing.T) {
_, err := Load(`a = 0k1`)
if err.Error() != "(1, 5): unknown number base: k. possible options are x (hex) o (octal) b (binary)" {
t.Error("Error should have been returned.")
}
}
func TestIntegerNoDigit(t *testing.T) {
_, err := Load(`a = 0b`)
if err.Error() != "(1, 5): number needs at least one digit" {
t.Error("Bad error message:", err.Error())
}
}
func TestNumbersWithUnderscores(t *testing.T) { func TestNumbersWithUnderscores(t *testing.T) {
tree, err := Load("a = 1_000") tree, err := Load("a = 1_000")
assertTree(t, tree, err, map[string]interface{}{ assertTree(t, tree, err, map[string]interface{}{
@@ -155,6 +239,36 @@ func TestSpaceKey(t *testing.T) {
}) })
} }
func TestDoubleQuotedKey(t *testing.T) {
tree, err := Load(`
"key" = "a"
"\t" = "b"
"\U0001F914" = "c"
"\u2764" = "d"
`)
assertTree(t, tree, err, map[string]interface{}{
"key": "a",
"\t": "b",
"\U0001F914": "c",
"\u2764": "d",
})
}
func TestSingleQuotedKey(t *testing.T) {
tree, err := Load(`
'key' = "a"
'\t' = "b"
'\U0001F914' = "c"
'\u2764' = "d"
`)
assertTree(t, tree, err, map[string]interface{}{
`key`: "a",
`\t`: "b",
`\U0001F914`: "c",
`\u2764`: "d",
})
}
func TestStringEscapables(t *testing.T) { func TestStringEscapables(t *testing.T) {
tree, err := Load("a = \"a \\n b\"") tree, err := Load("a = \"a \\n b\"")
assertTree(t, tree, err, map[string]interface{}{ assertTree(t, tree, err, map[string]interface{}{
@@ -642,7 +756,7 @@ func TestTomlValueStringRepresentation(t *testing.T) {
{int64(12345), "12345"}, {int64(12345), "12345"},
{uint64(50), "50"}, {uint64(50), "50"},
{float64(123.45), "123.45"}, {float64(123.45), "123.45"},
{bool(true), "true"}, {true, "true"},
{"hello world", "\"hello world\""}, {"hello world", "\"hello world\""},
{"\b\t\n\f\r\"\\", "\"\\b\\t\\n\\f\\r\\\"\\\\\""}, {"\b\t\n\f\r\"\\", "\"\\b\\t\\n\\f\\r\\\"\\\\\""},
{"\x05", "\"\\u0005\""}, {"\x05", "\"\\u0005\""},
@@ -652,7 +766,7 @@ func TestTomlValueStringRepresentation(t *testing.T) {
"[\"gamma\",\"delta\"]"}, "[\"gamma\",\"delta\"]"},
{nil, ""}, {nil, ""},
} { } {
result, err := tomlValueStringRepresentation(item.Value) result, err := tomlValueStringRepresentation(item.Value, "", false)
if err != nil { if err != nil {
t.Errorf("Test %d - unexpected error: %s", idx, err) t.Errorf("Test %d - unexpected error: %s", idx, err)
} }
+1 -1
View File
@@ -139,7 +139,7 @@
// Compiled Queries // Compiled Queries
// //
// Queries may be executed directly on a Tree object, or compiled ahead // Queries may be executed directly on a Tree object, or compiled ahead
// of time and executed discretely. The former is more convienent, but has the // of time and executed discretely. The former is more convenient, but has the
// penalty of having to recompile the query expression each time. // penalty of having to recompile the query expression each time.
// //
// // basic query // // basic query
+3 -3
View File
@@ -2,12 +2,13 @@ package query
import ( import (
"fmt" "fmt"
"github.com/pelletier/go-toml"
"io/ioutil" "io/ioutil"
"sort" "sort"
"strings" "strings"
"testing" "testing"
"time" "time"
"github.com/pelletier/go-toml"
) )
type queryTestNode struct { type queryTestNode struct {
@@ -406,8 +407,7 @@ func TestQueryFilterFn(t *testing.T) {
assertQueryPositions(t, string(buff), assertQueryPositions(t, string(buff),
"$..[?(float)]", "$..[?(float)]",
[]interface{}{ []interface{}{ // no float values in document
// no float values in document
}) })
tv, _ := time.Parse(time.RFC3339, "1979-05-27T07:32:00Z") tv, _ := time.Parse(time.RFC3339, "1979-05-27T07:32:00Z")
+1 -3
View File
@@ -1,6 +1,7 @@
#!/bin/bash #!/bin/bash
# fail out of the script if anything here fails # fail out of the script if anything here fails
set -e set -e
set -o pipefail
# set the path to the present working directory # set the path to the present working directory
export GOPATH=`pwd` export GOPATH=`pwd`
@@ -22,9 +23,6 @@ function git_clone() {
# Remove potential previous runs # Remove potential previous runs
rm -rf src test_program_bin toml-test rm -rf src test_program_bin toml-test
# Run go vet
go vet ./...
go get github.com/pelletier/go-buffruneio go get github.com/pelletier/go-buffruneio
go get github.com/davecgh/go-spew/spew go get github.com/davecgh/go-spew/spew
go get gopkg.in/yaml.v2 go get gopkg.in/yaml.v2
+4
View File
@@ -23,6 +23,8 @@ const (
tokenTrue tokenTrue
tokenFalse tokenFalse
tokenFloat tokenFloat
tokenInf
tokenNan
tokenEqual tokenEqual
tokenLeftBracket tokenLeftBracket
tokenRightBracket tokenRightBracket
@@ -55,6 +57,8 @@ var tokenTypeNames = []string{
"True", "True",
"False", "False",
"Float", "Float",
"Inf",
"NaN",
"=", "=",
"[", "[",
"]", "]",
+96 -21
View File
@@ -11,14 +11,19 @@ import (
) )
type tomlValue struct { type tomlValue struct {
value interface{} // string, int64, uint64, float64, bool, time.Time, [] of any of this list value interface{} // string, int64, uint64, float64, bool, time.Time, [] of any of this list
position Position comment string
commented bool
multiline bool
position Position
} }
// Tree is the result of the parsing of a TOML file. // Tree is the result of the parsing of a TOML file.
type Tree struct { type Tree struct {
values map[string]interface{} // string -> *tomlValue, *Tree, []*Tree values map[string]interface{} // string -> *tomlValue, *Tree, []*Tree
position Position comment string
commented bool
position Position
} }
func newTree() *Tree { func newTree() *Tree {
@@ -67,18 +72,15 @@ func (t *Tree) Keys() []string {
} }
// Get the value at key in the Tree. // Get the value at key in the Tree.
// Key is a dot-separated path (e.g. a.b.c). // Key is a dot-separated path (e.g. a.b.c) without single/double quoted strings.
// If you need to retrieve non-bare keys, use GetPath.
// Returns nil if the path does not exist in the tree. // Returns nil if the path does not exist in the tree.
// If keys is of length zero, the current tree is returned. // If keys is of length zero, the current tree is returned.
func (t *Tree) Get(key string) interface{} { func (t *Tree) Get(key string) interface{} {
if key == "" { if key == "" {
return t return t
} }
comps, err := parseKey(key) return t.GetPath(strings.Split(key, "."))
if err != nil {
return nil
}
return t.GetPath(comps)
} }
// GetPath returns the element in the tree indicated by 'keys'. // GetPath returns the element in the tree indicated by 'keys'.
@@ -174,17 +176,23 @@ func (t *Tree) GetDefault(key string, def interface{}) interface{} {
return val return val
} }
// Set an element in the tree. // SetOptions arguments are supplied to the SetWithOptions and SetPathWithOptions functions to modify marshalling behaviour.
// Key is a dot-separated path (e.g. a.b.c). // The default values within the struct are valid default options.
// Creates all necessary intermediate trees, if needed. type SetOptions struct {
func (t *Tree) Set(key string, value interface{}) { Comment string
t.SetPath(strings.Split(key, "."), value) Commented bool
Multiline bool
} }
// SetPath sets an element in the tree. // SetWithOptions is the same as Set, but allows you to provide formatting
// Keys is an array of path elements (e.g. {"a","b","c"}). // instructions to the key, that will be used by Marshal().
// Creates all necessary intermediate trees, if needed. func (t *Tree) SetWithOptions(key string, opts SetOptions, value interface{}) {
func (t *Tree) SetPath(keys []string, value interface{}) { t.SetPathWithOptions(strings.Split(key, "."), opts, value)
}
// SetPathWithOptions is the same as SetPath, but allows you to provide
// formatting instructions to the key, that will be reused by Marshal().
func (t *Tree) SetPathWithOptions(keys []string, opts SetOptions, value interface{}) {
subtree := t subtree := t
for _, intermediateKey := range keys[:len(keys)-1] { for _, intermediateKey := range keys[:len(keys)-1] {
nextTree, exists := subtree.values[intermediateKey] nextTree, exists := subtree.values[intermediateKey]
@@ -209,13 +217,80 @@ func (t *Tree) SetPath(keys []string, value interface{}) {
switch value.(type) { switch value.(type) {
case *Tree: case *Tree:
tt := value.(*Tree)
tt.comment = opts.Comment
toInsert = value toInsert = value
case []*Tree: case []*Tree:
toInsert = value toInsert = value
case *tomlValue: case *tomlValue:
toInsert = value tt := value.(*tomlValue)
tt.comment = opts.Comment
toInsert = tt
default: default:
toInsert = &tomlValue{value: value} toInsert = &tomlValue{value: value, comment: opts.Comment, commented: opts.Commented, multiline: opts.Multiline}
}
subtree.values[keys[len(keys)-1]] = toInsert
}
// Set an element in the tree.
// Key is a dot-separated path (e.g. a.b.c).
// Creates all necessary intermediate trees, if needed.
func (t *Tree) Set(key string, value interface{}) {
t.SetWithComment(key, "", false, value)
}
// SetWithComment is the same as Set, but allows you to provide comment
// information to the key, that will be reused by Marshal().
func (t *Tree) SetWithComment(key string, comment string, commented bool, value interface{}) {
t.SetPathWithComment(strings.Split(key, "."), comment, commented, value)
}
// SetPath sets an element in the tree.
// Keys is an array of path elements (e.g. {"a","b","c"}).
// Creates all necessary intermediate trees, if needed.
func (t *Tree) SetPath(keys []string, value interface{}) {
t.SetPathWithComment(keys, "", false, value)
}
// SetPathWithComment is the same as SetPath, but allows you to provide comment
// information to the key, that will be reused by Marshal().
func (t *Tree) SetPathWithComment(keys []string, comment string, commented bool, value interface{}) {
subtree := t
for _, intermediateKey := range keys[:len(keys)-1] {
nextTree, exists := subtree.values[intermediateKey]
if !exists {
nextTree = newTree()
subtree.values[intermediateKey] = nextTree // add new element here
}
switch node := nextTree.(type) {
case *Tree:
subtree = node
case []*Tree:
// go to most recent element
if len(node) == 0 {
// create element if it does not exist
subtree.values[intermediateKey] = append(node, newTree())
}
subtree = node[len(node)-1]
}
}
var toInsert interface{}
switch value.(type) {
case *Tree:
tt := value.(*Tree)
tt.comment = comment
toInsert = value
case []*Tree:
toInsert = value
case *tomlValue:
tt := value.(*tomlValue)
tt.comment = comment
toInsert = tt
default:
toInsert = &tomlValue{value: value, comment: comment, commented: commented}
} }
subtree.values[keys[len(keys)-1]] = toInsert subtree.values[keys[len(keys)-1]] = toInsert
+3 -3
View File
@@ -104,7 +104,7 @@ func sliceToTree(object interface{}) (interface{}, error) {
} }
arrayValue = reflect.Append(arrayValue, reflect.ValueOf(simpleValue)) arrayValue = reflect.Append(arrayValue, reflect.ValueOf(simpleValue))
} }
return &tomlValue{arrayValue.Interface(), Position{}}, nil return &tomlValue{value: arrayValue.Interface(), position: Position{}}, nil
} }
func toTree(object interface{}) (interface{}, error) { func toTree(object interface{}) (interface{}, error) {
@@ -127,7 +127,7 @@ func toTree(object interface{}) (interface{}, error) {
} }
values[key.String()] = newValue values[key.String()] = newValue
} }
return &Tree{values, Position{}}, nil return &Tree{values: values, position: Position{}}, nil
} }
if value.Kind() == reflect.Array || value.Kind() == reflect.Slice { if value.Kind() == reflect.Array || value.Kind() == reflect.Slice {
@@ -138,5 +138,5 @@ func toTree(object interface{}) (interface{}, error) {
if err != nil { if err != nil {
return nil, err return nil, err
} }
return &tomlValue{simpleValue, Position{}}, nil return &tomlValue{value: simpleValue, position: Position{}}, nil
} }
+2 -2
View File
@@ -60,7 +60,7 @@ func TestTreeCreateToTree(t *testing.T) {
}, },
"array": []string{"a", "b", "c"}, "array": []string{"a", "b", "c"},
"array_uint": []uint{uint(1), uint(2)}, "array_uint": []uint{uint(1), uint(2)},
"array_table": []map[string]interface{}{map[string]interface{}{"sub_map": 52}}, "array_table": []map[string]interface{}{{"sub_map": 52}},
"array_times": []time.Time{time.Now(), time.Now()}, "array_times": []time.Time{time.Now(), time.Now()},
"map_times": map[string]time.Time{"now": time.Now()}, "map_times": map[string]time.Time{"now": time.Now()},
"custom_string_map_key": map[customString]interface{}{customString("custom"): "custom"}, "custom_string_map_key": map[customString]interface{}{customString("custom"): "custom"},
@@ -97,7 +97,7 @@ func TestTreeCreateToTreeInvalidArrayMemberType(t *testing.T) {
} }
func TestTreeCreateToTreeInvalidTableGroupType(t *testing.T) { func TestTreeCreateToTreeInvalidTableGroupType(t *testing.T) {
_, err := TreeFromMap(map[string]interface{}{"foo": []map[string]interface{}{map[string]interface{}{"hello": t}}}) _, err := TreeFromMap(map[string]interface{}{"foo": []map[string]interface{}{{"hello": t}}})
expected := "cannot convert type *testing.T to Tree" expected := "cannot convert type *testing.T to Tree"
if err.Error() != expected { if err.Error() != expected {
t.Fatalf("expected error %s, got %s", expected, err.Error()) t.Fatalf("expected error %s, got %s", expected, err.Error())
+115 -15
View File
@@ -12,7 +12,41 @@ import (
"time" "time"
) )
// encodes a string to a TOML-compliant string value // Encodes a string to a TOML-compliant multi-line string value
// This function is a clone of the existing encodeTomlString function, except that whitespace characters
// are preserved. Quotation marks and backslashes are also not escaped.
func encodeMultilineTomlString(value string) string {
var b bytes.Buffer
for _, rr := range value {
switch rr {
case '\b':
b.WriteString(`\b`)
case '\t':
b.WriteString("\t")
case '\n':
b.WriteString("\n")
case '\f':
b.WriteString(`\f`)
case '\r':
b.WriteString("\r")
case '"':
b.WriteString(`"`)
case '\\':
b.WriteString(`\`)
default:
intRr := uint16(rr)
if intRr < 0x001F {
b.WriteString(fmt.Sprintf("\\u%0.4X", intRr))
} else {
b.WriteRune(rr)
}
}
}
return b.String()
}
// Encodes a string to a TOML-compliant string value
func encodeTomlString(value string) string { func encodeTomlString(value string) string {
var b bytes.Buffer var b bytes.Buffer
@@ -44,7 +78,16 @@ func encodeTomlString(value string) string {
return b.String() return b.String()
} }
func tomlValueStringRepresentation(v interface{}) (string, error) { func tomlValueStringRepresentation(v interface{}, indent string, arraysOneElementPerLine bool) (string, error) {
// this interface check is added to dereference the change made in the writeTo function.
// That change was made to allow this function to see formatting options.
tv, ok := v.(*tomlValue)
if ok {
v = tv.value
} else {
tv = &tomlValue{}
}
switch value := v.(type) { switch value := v.(type) {
case uint64: case uint64:
return strconv.FormatUint(value, 10), nil return strconv.FormatUint(value, 10), nil
@@ -54,14 +97,17 @@ func tomlValueStringRepresentation(v interface{}) (string, error) {
// Ensure a round float does contain a decimal point. Otherwise feeding // Ensure a round float does contain a decimal point. Otherwise feeding
// the output back to the parser would convert to an integer. // the output back to the parser would convert to an integer.
if math.Trunc(value) == value { if math.Trunc(value) == value {
return strconv.FormatFloat(value, 'f', 1, 32), nil return strings.ToLower(strconv.FormatFloat(value, 'f', 1, 32)), nil
} }
return strconv.FormatFloat(value, 'f', -1, 32), nil return strings.ToLower(strconv.FormatFloat(value, 'f', -1, 32)), nil
case string: case string:
if tv.multiline {
return "\"\"\"\n" + encodeMultilineTomlString(value) + "\"\"\"", nil
}
return "\"" + encodeTomlString(value) + "\"", nil return "\"" + encodeTomlString(value) + "\"", nil
case []byte: case []byte:
b, _ := v.([]byte) b, _ := v.([]byte)
return tomlValueStringRepresentation(string(b)) return tomlValueStringRepresentation(string(b), indent, arraysOneElementPerLine)
case bool: case bool:
if value { if value {
return "true", nil return "true", nil
@@ -76,21 +122,38 @@ func tomlValueStringRepresentation(v interface{}) (string, error) {
rv := reflect.ValueOf(v) rv := reflect.ValueOf(v)
if rv.Kind() == reflect.Slice { if rv.Kind() == reflect.Slice {
values := []string{} var values []string
for i := 0; i < rv.Len(); i++ { for i := 0; i < rv.Len(); i++ {
item := rv.Index(i).Interface() item := rv.Index(i).Interface()
itemRepr, err := tomlValueStringRepresentation(item) itemRepr, err := tomlValueStringRepresentation(item, indent, arraysOneElementPerLine)
if err != nil { if err != nil {
return "", err return "", err
} }
values = append(values, itemRepr) values = append(values, itemRepr)
} }
if arraysOneElementPerLine && len(values) > 1 {
stringBuffer := bytes.Buffer{}
valueIndent := indent + ` ` // TODO: move that to a shared encoder state
stringBuffer.WriteString("[\n")
for _, value := range values {
stringBuffer.WriteString(valueIndent)
stringBuffer.WriteString(value)
stringBuffer.WriteString(`,`)
stringBuffer.WriteString("\n")
}
stringBuffer.WriteString(indent + "]")
return stringBuffer.String(), nil
}
return "[" + strings.Join(values, ",") + "]", nil return "[" + strings.Join(values, ",") + "]", nil
} }
return "", fmt.Errorf("unsupported value type %T: %v", v, v) return "", fmt.Errorf("unsupported value type %T: %v", v, v)
} }
func (t *Tree) writeTo(w io.Writer, indent, keyspace string, bytesCount int64) (int64, error) { func (t *Tree) writeTo(w io.Writer, indent, keyspace string, bytesCount int64, arraysOneElementPerLine bool) (int64, error) {
simpleValuesKeys := make([]string, 0) simpleValuesKeys := make([]string, 0)
complexValuesKeys := make([]string, 0) complexValuesKeys := make([]string, 0)
@@ -113,12 +176,29 @@ func (t *Tree) writeTo(w io.Writer, indent, keyspace string, bytesCount int64) (
return bytesCount, fmt.Errorf("invalid value type at %s: %T", k, t.values[k]) return bytesCount, fmt.Errorf("invalid value type at %s: %T", k, t.values[k])
} }
repr, err := tomlValueStringRepresentation(v.value) repr, err := tomlValueStringRepresentation(v, indent, arraysOneElementPerLine)
if err != nil { if err != nil {
return bytesCount, err return bytesCount, err
} }
writtenBytesCount, err := writeStrings(w, indent, k, " = ", repr, "\n") if v.comment != "" {
comment := strings.Replace(v.comment, "\n", "\n"+indent+"#", -1)
start := "# "
if strings.HasPrefix(comment, "#") {
start = ""
}
writtenBytesCountComment, errc := writeStrings(w, "\n", indent, start, comment, "\n")
bytesCount += int64(writtenBytesCountComment)
if errc != nil {
return bytesCount, errc
}
}
var commented string
if v.commented {
commented = "# "
}
writtenBytesCount, err := writeStrings(w, indent, commented, k, " = ", repr, "\n")
bytesCount += int64(writtenBytesCount) bytesCount += int64(writtenBytesCount)
if err != nil { if err != nil {
return bytesCount, err return bytesCount, err
@@ -132,28 +212,48 @@ func (t *Tree) writeTo(w io.Writer, indent, keyspace string, bytesCount int64) (
if keyspace != "" { if keyspace != "" {
combinedKey = keyspace + "." + combinedKey combinedKey = keyspace + "." + combinedKey
} }
var commented string
if t.commented {
commented = "# "
}
switch node := v.(type) { switch node := v.(type) {
// node has to be of those two types given how keys are sorted above // node has to be of those two types given how keys are sorted above
case *Tree: case *Tree:
writtenBytesCount, err := writeStrings(w, "\n", indent, "[", combinedKey, "]\n") tv, ok := t.values[k].(*Tree)
if !ok {
return bytesCount, fmt.Errorf("invalid value type at %s: %T", k, t.values[k])
}
if tv.comment != "" {
comment := strings.Replace(tv.comment, "\n", "\n"+indent+"#", -1)
start := "# "
if strings.HasPrefix(comment, "#") {
start = ""
}
writtenBytesCountComment, errc := writeStrings(w, "\n", indent, start, comment)
bytesCount += int64(writtenBytesCountComment)
if errc != nil {
return bytesCount, errc
}
}
writtenBytesCount, err := writeStrings(w, "\n", indent, commented, "[", combinedKey, "]\n")
bytesCount += int64(writtenBytesCount) bytesCount += int64(writtenBytesCount)
if err != nil { if err != nil {
return bytesCount, err return bytesCount, err
} }
bytesCount, err = node.writeTo(w, indent+" ", combinedKey, bytesCount) bytesCount, err = node.writeTo(w, indent+" ", combinedKey, bytesCount, arraysOneElementPerLine)
if err != nil { if err != nil {
return bytesCount, err return bytesCount, err
} }
case []*Tree: case []*Tree:
for _, subTree := range node { for _, subTree := range node {
writtenBytesCount, err := writeStrings(w, "\n", indent, "[[", combinedKey, "]]\n") writtenBytesCount, err := writeStrings(w, "\n", indent, commented, "[[", combinedKey, "]]\n")
bytesCount += int64(writtenBytesCount) bytesCount += int64(writtenBytesCount)
if err != nil { if err != nil {
return bytesCount, err return bytesCount, err
} }
bytesCount, err = subTree.writeTo(w, indent+" ", combinedKey, bytesCount) bytesCount, err = subTree.writeTo(w, indent+" ", combinedKey, bytesCount, arraysOneElementPerLine)
if err != nil { if err != nil {
return bytesCount, err return bytesCount, err
} }
@@ -179,7 +279,7 @@ func writeStrings(w io.Writer, s ...string) (int, error) {
// WriteTo encode the Tree as Toml and writes it to the writer w. // WriteTo encode the Tree as Toml and writes it to the writer w.
// Returns the number of bytes written in case of success, or an error if anything happened. // Returns the number of bytes written in case of success, or an error if anything happened.
func (t *Tree) WriteTo(w io.Writer) (int64, error) { func (t *Tree) WriteTo(w io.Writer) (int64, error) {
return t.writeTo(w, "", "", 0) return t.writeTo(w, "", "", 0, false)
} }
// ToTomlString generates a human-readable representation of the current tree. // ToTomlString generates a human-readable representation of the current tree.
+26 -8
View File
@@ -30,7 +30,7 @@ func (f *failingWriter) Write(p []byte) (n int, err error) {
f.buffer.Write(p[:toWrite]) f.buffer.Write(p[:toWrite])
f.written = f.failAt f.written = f.failAt
return toWrite, fmt.Errorf("failingWriter failed after writting %d bytes", f.written) return toWrite, fmt.Errorf("failingWriter failed after writing %d bytes", f.written)
} }
func assertErrorString(t *testing.T, expected string, err error) { func assertErrorString(t *testing.T, expected string, err error) {
@@ -161,13 +161,13 @@ func TestTreeWriteToInvalidTreeSimpleValue(t *testing.T) {
} }
func TestTreeWriteToInvalidTreeTomlValue(t *testing.T) { func TestTreeWriteToInvalidTreeTomlValue(t *testing.T) {
tree := Tree{values: map[string]interface{}{"foo": &tomlValue{int8(1), Position{}}}} tree := Tree{values: map[string]interface{}{"foo": &tomlValue{value: int8(1), comment: "", position: Position{}}}}
_, err := tree.ToTomlString() _, err := tree.ToTomlString()
assertErrorString(t, "unsupported value type int8: 1", err) assertErrorString(t, "unsupported value type int8: 1", err)
} }
func TestTreeWriteToInvalidTreeTomlValueArray(t *testing.T) { func TestTreeWriteToInvalidTreeTomlValueArray(t *testing.T) {
tree := Tree{values: map[string]interface{}{"foo": &tomlValue{[]interface{}{int8(1)}, Position{}}}} tree := Tree{values: map[string]interface{}{"foo": &tomlValue{value: int8(1), comment: "", position: Position{}}}}
_, err := tree.ToTomlString() _, err := tree.ToTomlString()
assertErrorString(t, "unsupported value type int8: 1", err) assertErrorString(t, "unsupported value type int8: 1", err)
} }
@@ -176,7 +176,7 @@ func TestTreeWriteToFailingWriterInSimpleValue(t *testing.T) {
toml, _ := Load(`a = 2`) toml, _ := Load(`a = 2`)
writer := failingWriter{failAt: 0, written: 0} writer := failingWriter{failAt: 0, written: 0}
_, err := toml.WriteTo(&writer) _, err := toml.WriteTo(&writer)
assertErrorString(t, "failingWriter failed after writting 0 bytes", err) assertErrorString(t, "failingWriter failed after writing 0 bytes", err)
} }
func TestTreeWriteToFailingWriterInTable(t *testing.T) { func TestTreeWriteToFailingWriterInTable(t *testing.T) {
@@ -185,11 +185,11 @@ func TestTreeWriteToFailingWriterInTable(t *testing.T) {
a = 2`) a = 2`)
writer := failingWriter{failAt: 2, written: 0} writer := failingWriter{failAt: 2, written: 0}
_, err := toml.WriteTo(&writer) _, err := toml.WriteTo(&writer)
assertErrorString(t, "failingWriter failed after writting 2 bytes", err) assertErrorString(t, "failingWriter failed after writing 2 bytes", err)
writer = failingWriter{failAt: 13, written: 0} writer = failingWriter{failAt: 13, written: 0}
_, err = toml.WriteTo(&writer) _, err = toml.WriteTo(&writer)
assertErrorString(t, "failingWriter failed after writting 13 bytes", err) assertErrorString(t, "failingWriter failed after writing 13 bytes", err)
} }
func TestTreeWriteToFailingWriterInArray(t *testing.T) { func TestTreeWriteToFailingWriterInArray(t *testing.T) {
@@ -198,11 +198,11 @@ func TestTreeWriteToFailingWriterInArray(t *testing.T) {
a = 2`) a = 2`)
writer := failingWriter{failAt: 2, written: 0} writer := failingWriter{failAt: 2, written: 0}
_, err := toml.WriteTo(&writer) _, err := toml.WriteTo(&writer)
assertErrorString(t, "failingWriter failed after writting 2 bytes", err) assertErrorString(t, "failingWriter failed after writing 2 bytes", err)
writer = failingWriter{failAt: 15, written: 0} writer = failingWriter{failAt: 15, written: 0}
_, err = toml.WriteTo(&writer) _, err = toml.WriteTo(&writer)
assertErrorString(t, "failingWriter failed after writting 15 bytes", err) assertErrorString(t, "failingWriter failed after writing 15 bytes", err)
} }
func TestTreeWriteToMapExampleFile(t *testing.T) { func TestTreeWriteToMapExampleFile(t *testing.T) {
@@ -309,6 +309,24 @@ func TestTreeWriteToFloat(t *testing.T) {
} }
} }
func TestTreeWriteToSpecialFloat(t *testing.T) {
expected := `a = +inf
b = -inf
c = nan`
tree, err := Load(expected)
if err != nil {
t.Fatal(err)
}
str, err := tree.ToTomlString()
if err != nil {
t.Fatal(err)
}
if strings.TrimSpace(str) != strings.TrimSpace(expected) {
t.Fatalf("Expected:\n%s\nGot:\n%s", expected, str)
}
}
func BenchmarkTreeToTomlString(b *testing.B) { func BenchmarkTreeToTomlString(b *testing.B) {
toml, err := Load(sampleHard) toml, err := Load(sampleHard)
if err != nil { if err != nil {