Thread byte offset information through all error creation sites,
eliminating the need for SubsliceOffset to recover position from
pointer comparison.
Changes:
- Add Offset field to ParserError struct
- Add offset parameter to NewParserError
- Add Parser.offsetOf helper for suffix-length arithmetic
- Thread base offset through scanner functions (scanComment,
scanBasicString, scanMultilineBasicString, scanLiteralString,
scanMultilineLiteralString, scanWindowsNewline)
- Thread base offset through standalone functions (expect, hexToRune)
- Thread base offset through all decode functions (parseInteger,
parseFloat, parseLocalDate, parseLocalTime, parseLocalDateTime,
parseDateTime, checkAndRemoveUnderscores*)
- Update all unmarshaler call sites to pass value.Raw.Offset
- Update localtime.go UnmarshalText methods with base=0
- Update strict.go to populate Offset from key ranges
- Change wrapDecodeError to read de.Offset directly
- Change Utf8TomlValidAlreadyEscaped to return int index (-1 if valid)
instead of a byte subslice
- Unexport SubsliceOffset (now only used internally by Range())
This makes error positions self-describing: each ParserError carries its
own byte offset, so callers no longer need the original document slice
and address arithmetic to determine where an error occurred.
Co-authored-by: Thomas Pelletier <thomas@pelletier.dev>
Fixes#873
Extend the unstable.Unmarshaler interface support to work with tables
and array tables, not just single values.
When a type implementing unstable.Unmarshaler is the target of a table
(e.g., [table] or [[array]]), the UnmarshalTOML method receives a
synthetic InlineTable node containing all the key-value pairs belonging
to that table.
Key changes:
- Add handleKeyValuesUnmarshaler to collect and process table content
- Add copyExpressionNodes to deep-copy AST nodes for synthetic tables
- Add helper functions in unstable/ast.go for node manipulation
- Update documentation for EnableUnmarshalerInterface
- Add comprehensive tests for table and array table unmarshaling
* Implement bytes-based Unmarshaler interface for tables and arrays (#873)
This change brings back support for the unstable.Unmarshaler interface
for tables and array tables, addressing issue #873.
Key changes:
- Changed UnmarshalTOML signature from (*Node) to ([]byte) to provide
raw TOML bytes instead of AST nodes
- Added RawMessage type (similar to json.RawMessage) for capturing raw
TOML bytes for later processing
- Updated handleKeyValuesUnmarshaler to reconstruct key-value lines
from the parsed keys and raw value bytes
- Added support for slice types implementing Unmarshaler (e.g., RawMessage)
- Removed unused AST helper functions from unstable/ast.go
The bytes-based interface allows users to:
- Get raw TOML bytes for custom parsing
- Delay TOML decoding using RawMessage
- Implement custom unmarshaling logic for complex types
Tests added for:
- Table unmarshaler with various scenarios
- Array table unmarshaler
- Split tables (same parent defined in multiple places)
- RawMessage usage
- Nested tables and mixed regular fields
* Fix lint issues and improve test coverage for Unmarshaler interface
- Apply De Morgan's law in keyNeedsQuoting to satisfy staticcheck QF1001
- Remove unused splitTableUnmarshaler type from test
- Fix unused parameter lint warning in errorUnmarshaler873
- Add test for quoted keys that need special handling
- Add test for error propagation from UnmarshalTOML
- Update customTable873 parser to handle quoted keys properly
Coverage improved:
- handleKeyValuesUnmarshaler: 80.0% -> 93.3%
- keyNeedsQuoting: 66.7% -> 83.3%
- Overall main package: 97.2% -> 97.5%
* Add test for dotted keys to improve coverage
Add TestIssue873_DottedKeys to test dotted key handling (e.g., sub.key = value)
in the Unmarshaler interface. This improves coverage for handleKeyValuesUnmarshaler
from 93.3% to 96.7%.
* Add double pointer test to achieve 100% coverage for handleKeyValues
Add TestIssue873_DoublePointerUnmarshaler to test pointer-to-pointer
to Unmarshaler types. This covers the pointer dereferencing loop in
handleKeyValues, bringing its coverage from 88% to 100%.
Total coverage: 97.4%
* Add Example tests and fix raw value extraction for boolean types
Add two godoc Example tests:
- ExampleDecoder_EnableUnmarshalerInterface_dynamicConfig: shows dynamic
unmarshaling based on a type field
- ExampleDecoder_EnableUnmarshalerInterface_rawMessage: demonstrates
RawMessage usage for deferred parsing
Fix handleKeyValuesUnmarshaler to handle values where Raw.Length == 0
(like boolean types) by using value.Data as fallback.
* Preserve original formatting in Unmarshaler by using raw byte ranges
Instead of reconstructing key-value lines from parsed components, now
uses the original raw bytes from the document. This preserves:
- Whitespace around '=' (e.g., "key = value")
- String quoting style (basic vs literal)
- Number formats (hex, octal, binary)
- Inline table formatting
Changes:
- Add Raw range tracking to KeyValue expressions in parseKeyval
- Update handleKeyValuesUnmarshaler to use expr.Raw directly
- Remove keyNeedsQuoting helper (no longer needed)
- Add TestIssue873_FormattingPreservation test
- Update expected output in ExampleParser_comments
* Prevent test matrix from canceling on first failure
Add fail-fast: false to the test workflow strategy so that all
OS/Go version combinations continue running even if one fails.
This provides better visibility into which specific combinations
have issues.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Return a type mismatch error instead of panicking when datetime values
(DateTime, LocalDate, LocalTime, LocalDateTime) are unmarshaled into
incompatible Go types. This makes the decoder safer for processing
untrusted TOML input.
https://claude.ai/code/session_011jwvtDS5M2KncLrqJpgMr5
Co-authored-by: Claude <noreply@anthropic.com>
When a type implements encoding.TextUnmarshaler, the unmarshaler now
skips calling UnmarshalText for Array and InlineTable TOML values.
This allows types to support both:
- Simple string values via UnmarshalText
- Structured table values via field-by-field unmarshaling
Previously, UnmarshalText was called unconditionally, which prevented
proper struct unmarshaling when the TOML value was a table or array
of tables.
Co-authored-by: Claude <noreply@anthropic.com>
Removes all unsafe operations from go-toml, making the codebase
fully safe Go code. The internal/danger package that contained
unsafe operations has been deleted.
Changes:
- Replace pointer-based node navigation with index-based navigation
- Node.next and Node.child now store absolute indices into the
backing nodes slice instead of relative offsets
- Add nodes pointer to Node and Iterator for safe navigation
- Replace danger.TypeID with reflect.Type for cache keys
- Delete internal/danger package entirely
Performance overhead is under 10% compared to the unsafe version,
which is acceptable for the safety and maintainability benefits.
[Cursor][claude-sonnet-4-20250514]
Recent versions of Go object to the use of non-constant variables a
format strings. This commit fixes errors like this:
cli.go:26:47: non-constant format string in call to fmt.Fprintf
Signed-off-by: W. Michael Petullo <mike@flyn.org>
This way, calls to Unmarshal or Decoder.Decode allocate once
at the start rather than twice.
│ old │ new │
│ allocs/op │ allocs/op vs base │
Unmarshal/HugoFrontMatter-8 141.0 ± 0% 140.0 ± 0% -0.71% (p=0.002 n=6)
* go.mod: bump minimum and language to 1.21
CI only tests Go 1.21 and 1.22, and older versions of Go are no longer
getting any bug or security fixes, so advertise that we only support
Go 1.21 or later via go.mod.
While here, ensure the module is tidy and resolve deprecation warnings,
and remove now-unnecessary Go version build tags.
* replace sort.Slice with slices.SortFunc
The latter is more efficient, and allocates less, since sort.Slice
needs to go through sort.Interface which causes allocations.
goos: linux
goarch: amd64
pkg: github.com/pelletier/go-toml/v2/benchmark
cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
│ old │ new │
│ sec/op │ sec/op vs base │
Marshal/HugoFrontMatter-8 7.612µ ± 1% 6.730µ ± 1% -11.59% (p=0.002 n=6)
│ old │ new │
│ B/s │ B/s vs base │
Marshal/HugoFrontMatter-8 65.52Mi ± 1% 74.11Mi ± 1% +13.11% (p=0.002 n=6)
│ old │ new │
│ B/op │ B/op vs base │
Marshal/HugoFrontMatter-8 5.672Ki ± 0% 5.266Ki ± 0% -7.16% (p=0.002 n=6)
│ old │ new │
│ allocs/op │ allocs/op vs base │
Marshal/HugoFrontMatter-8 85.00 ± 0% 73.00 ± 0% -14.12% (p=0.002 n=6)
When decoding into a non-empty slice, it needs to be emptied so that only the
tables contained in the document are present in the resulting value.
Arrays are not impacted because their unmarshal offset is tracked separately.
Fixes#931
Parser did not track the location of the faulty inline table in the
document, and unmarshaler tried to the use the non-raw data field of the
AST node, both resulting into a panic when generating the parser error.
Fixes#850
Uses the existing method to DRY up the error message generation, and decorates
with position index where needed. No behaviour is changed, but it allows for
further changes to make error messaging more specific.
Related to: pelletier/go-toml#806
As discussed[1], this change allows times to provide precision beyond the
nanosecond (nine digits fractional part). Extra precision is truncated according
to the TOML specificiation.
[1]: https://github.com/pelletier/go-toml/discussions/707
The goal is to provide some context as to why the type were mismatched. This
change only works for that case, on structs. This is the same a encoding/json. A
more general solution would be great, but this would require a broader change in
the decoder, which I don't think is necessary at the moment.
Fixes#628
The TOML spec is being clarified to say that dotted keys "define" their
intermediate tables. Therefore the seen tracker needs to verify that none of
them reference an explicit table.
Also added a missing seen expression check for key-values parsed as part of a
table section.
See https://github.com/toml-lang/toml/issues/846
Cursor Agent