1
0
mirror of https://github.com/kubernetes-sigs/descheduler.git synced 2026-01-26 13:29:11 +01:00

[v0.34.0] bump to kubernetes 1.34 deps

Signed-off-by: Amir Alavi <amiralavi7@gmail.com>
This commit is contained in:
Amir Alavi
2025-09-17 16:55:29 -04:00
parent e9188852ef
commit 1db6b615d1
1266 changed files with 100906 additions and 40660 deletions

View File

@@ -1,30 +1,31 @@
# CBOR Codec in Go
<!-- [![](https://github.com/fxamacker/images/raw/master/cbor/v2.5.0/fxamacker_cbor_banner.png)](#cbor-library-in-go) -->
<h1>CBOR Codec <a href="https://pkg.go.dev/github.com/fxamacker/cbor/v2"><img src="https://raw.githubusercontent.com/fxamacker/images/refs/heads/master/cbor/go-logo-blue.svg" alt="Go logo" style="height: 1em;" align="right"></a></h1>
[fxamacker/cbor](https://github.com/fxamacker/cbor) is a library for encoding and decoding [CBOR](https://www.rfc-editor.org/info/std94) and [CBOR Sequences](https://www.rfc-editor.org/rfc/rfc8742.html).
CBOR is a [trusted alternative](https://www.rfc-editor.org/rfc/rfc8949.html#name-comparison-of-other-binary-) to JSON, MessagePack, Protocol Buffers, etc.&nbsp; CBOR is an Internet&nbsp;Standard defined by [IETF&nbsp;STD&nbsp;94 (RFC&nbsp;8949)](https://www.rfc-editor.org/info/std94) and is designed to be relevant for decades.
`fxamacker/cbor` is used in projects by Arm Ltd., Cisco, EdgeX&nbsp;Foundry, Flow Foundation, Fraunhofer&#8209;AISEC, Kubernetes, Let's&nbsp;Encrypt (ISRG), Linux&nbsp;Foundation, Microsoft, Mozilla, Oasis&nbsp;Protocol, Tailscale, Teleport, [etc](https://github.com/fxamacker/cbor#who-uses-fxamackercbor).
`fxamacker/cbor` is used in projects by Arm Ltd., EdgeX&nbsp;Foundry, Flow Foundation, Fraunhofer&#8209;AISEC, IBM, Kubernetes[*](https://github.com/search?q=org%3Akubernetes%20fxamacker%2Fcbor&type=code), Let's&nbsp;Encrypt, Linux&nbsp;Foundation, Microsoft, Oasis&nbsp;Protocol, Red Hat[*](https://github.com/search?q=org%3Aopenshift+fxamacker%2Fcbor&type=code), Tailscale[*](https://github.com/search?q=org%3Atailscale+fxamacker%2Fcbor&type=code), Veraison[*](https://github.com/search?q=org%3Averaison+fxamacker%2Fcbor&type=code), [etc](https://github.com/fxamacker/cbor#who-uses-fxamackercbor).
See [Quick&nbsp;Start](#quick-start) and [Releases](https://github.com/fxamacker/cbor/releases/). 🆕 `UnmarshalFirst` and `DiagnoseFirst` can decode CBOR Sequences. `cbor.MarshalToBuffer()` and `UserBufferEncMode` accepts user-specified buffer.
See [Quick&nbsp;Start](#quick-start) and [Releases](https://github.com/fxamacker/cbor/releases/). 🆕 `UnmarshalFirst` and `DiagnoseFirst` can decode CBOR Sequences. `MarshalToBuffer` and `UserBufferEncMode` accepts user-specified buffer.
## fxamacker/cbor
[![](https://github.com/fxamacker/cbor/workflows/ci/badge.svg)](https://github.com/fxamacker/cbor/actions?query=workflow%3Aci)
[![](https://github.com/fxamacker/cbor/workflows/cover%20%E2%89%A596%25/badge.svg)](https://github.com/fxamacker/cbor/actions?query=workflow%3A%22cover+%E2%89%A596%25%22)
[![](https://github.com/fxamacker/cbor/workflows/cover%20%E2%89%A597%25/badge.svg)](https://github.com/fxamacker/cbor/actions?query=workflow%3A%22cover+%E2%89%A597%25%22)
[![CodeQL](https://github.com/fxamacker/cbor/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/fxamacker/cbor/actions/workflows/codeql-analysis.yml)
[![](https://img.shields.io/badge/fuzzing-passing-44c010)](#fuzzing-and-code-coverage)
[![Go Report Card](https://goreportcard.com/badge/github.com/fxamacker/cbor)](https://goreportcard.com/report/github.com/fxamacker/cbor)
[![](https://img.shields.io/ossf-scorecard/github.com/fxamacker/cbor?label=openssf%20scorecard)](https://github.com/fxamacker/cbor#fuzzing-and-code-coverage)
`fxamacker/cbor` is a CBOR codec in full conformance with [IETF STD&nbsp;94 (RFC&nbsp;8949)](https://www.rfc-editor.org/info/std94). It also supports CBOR Sequences ([RFC&nbsp;8742](https://www.rfc-editor.org/rfc/rfc8742.html)) and Extended Diagnostic Notation ([Appendix G of RFC&nbsp;8610](https://www.rfc-editor.org/rfc/rfc8610.html#appendix-G)).
Features include full support for CBOR tags, [Core Deterministic Encoding](https://www.rfc-editor.org/rfc/rfc8949.html#name-core-deterministic-encoding), duplicate map key detection, etc.
API is mostly same as `encoding/json`, plus interfaces that simplify concurrency and CBOR options.
Design balances trade-offs between security, speed, concurrency, encoded data size, usability, etc.
<details><summary>Highlights</summary><p/>
<details><summary> 🔎&nbsp; Highlights</summary><p/>
__🚀&nbsp; Speed__
@@ -38,7 +39,7 @@ Codec passed multiple confidential security assessments in 2022. No vulnerabili
__🗜️&nbsp; Data Size__
Struct tags (`toarray`, `keyasint`, `omitempty`) automatically reduce size of encoded structs. Encoding optionally shrinks float64→32→16 when values fit.
Struct tag options (`toarray`, `keyasint`, `omitempty`, `omitzero`) and field tag "-" automatically reduce size of encoded structs. Encoding optionally shrinks float64→32→16 when values fit.
__:jigsaw:&nbsp; Usability__
@@ -58,164 +59,205 @@ Features include CBOR [extension points](https://www.rfc-editor.org/rfc/rfc8949.
`fxamacker/cbor` has configurable limits, etc. that defend against malicious CBOR data.
By contrast, `encoding/gob` is [not designed to be hardened against adversarial inputs](https://pkg.go.dev/encoding/gob#hdr-Security).
Notably, `fxamacker/cbor` is fast at rejecting malformed CBOR data.
<details><summary>Example decoding with encoding/gob 💥 fatal error (out of memory)</summary><p/>
> [!NOTE]
> Benchmarks rejecting 10 bytes of malicious CBOR data decoding to `[]byte`:
>
> | Codec | Speed (ns/op) | Memory | Allocs |
> | :---- | ------------: | -----: | -----: |
> | fxamacker/cbor 2.7.0 | 47 ± 7% | 32 B/op | 2 allocs/op |
> | ugorji/go 1.2.12 | 5878187 ± 3% | 67111556 B/op | 13 allocs/op |
>
> Faster hardware (overclocked DDR4 or DDR5) can reduce speed difference.
>
> <details><summary> 🔎&nbsp; Benchmark details </summary><p/>
>
> Latest comparison for decoding CBOR data to Go `[]byte`:
> - Input: `[]byte{0x9B, 0x00, 0x00, 0x42, 0xFA, 0x42, 0xFA, 0x42, 0xFA, 0x42}`
> - go1.22.7, linux/amd64, i5-13600K (DDR4-2933, disabled e-cores)
> - go test -bench=. -benchmem -count=20
>
> #### Prior comparisons
>
> | Codec | Speed (ns/op) | Memory | Allocs |
> | :---- | ------------: | -----: | -----: |
> | fxamacker/cbor 2.5.0-beta2 | 44.33 ± 2% | 32 B/op | 2 allocs/op |
> | fxamacker/cbor 0.1.0 - 2.4.0 | ~44.68 ± 6% | 32 B/op | 2 allocs/op |
> | ugorji/go 1.2.10 | 5524792.50 ± 3% | 67110491 B/op | 12 allocs/op |
> | ugorji/go 1.1.0 - 1.2.6 | 💥 runtime: | out of memory: | cannot allocate |
>
> - Input: `[]byte{0x9B, 0x00, 0x00, 0x42, 0xFA, 0x42, 0xFA, 0x42, 0xFA, 0x42}`
> - go1.19.6, linux/amd64, i5-13600K (DDR4)
> - go test -bench=. -benchmem -count=20
>
> </details>
```Go
// Example of encoding/gob having "fatal error: runtime: out of memory"
// while decoding 181 bytes.
package main
import (
"bytes"
"encoding/gob"
"encoding/hex"
"fmt"
)
In contrast, some codecs can crash or use excessive resources while decoding bad data.
// Example data is from https://github.com/golang/go/issues/24446
// (shortened to 181 bytes).
const data = "4dffb503010102303001ff30000109010130010800010130010800010130" +
"01ffb80001014a01ffb60001014b01ff860001013001ff860001013001ff" +
"860001013001ff860001013001ffb80000001eff850401010e3030303030" +
"30303030303030303001ff3000010c0104000016ffb70201010830303030" +
"3030303001ff3000010c000030ffb6040405fcff00303030303030303030" +
"303030303030303030303030303030303030303030303030303030303030" +
"30"
> [!WARNING]
> Go's `encoding/gob` is [not designed to be hardened against adversarial inputs](https://pkg.go.dev/encoding/gob#hdr-Security).
>
> <details><summary> 🔎&nbsp; gob fatal error (out of memory) 💥 decoding 181 bytes</summary><p/>
>
> ```Go
> // Example of encoding/gob having "fatal error: runtime: out of memory"
> // while decoding 181 bytes (all Go versions as of Dec. 8, 2024).
> package main
> import (
> "bytes"
> "encoding/gob"
> "encoding/hex"
> "fmt"
> )
>
> // Example data is from https://github.com/golang/go/issues/24446
> // (shortened to 181 bytes).
> const data = "4dffb503010102303001ff30000109010130010800010130010800010130" +
> "01ffb80001014a01ffb60001014b01ff860001013001ff860001013001ff" +
> "860001013001ff860001013001ffb80000001eff850401010e3030303030" +
> "30303030303030303001ff3000010c0104000016ffb70201010830303030" +
> "3030303001ff3000010c000030ffb6040405fcff00303030303030303030" +
> "303030303030303030303030303030303030303030303030303030303030" +
> "30"
>
> type X struct {
> J *X
> K map[string]int
> }
>
> func main() {
> raw, _ := hex.DecodeString(data)
> decoder := gob.NewDecoder(bytes.NewReader(raw))
>
> var x X
> decoder.Decode(&x) // fatal error: runtime: out of memory
> fmt.Println("Decoding finished.")
> }
> ```
>
>
> </details>
type X struct {
J *X
K map[string]int
}
### Smaller Encodings with Struct Tag Options
func main() {
raw, _ := hex.DecodeString(data)
decoder := gob.NewDecoder(bytes.NewReader(raw))
Struct tags automatically reduce encoded size of structs and improve speed.
var x X
decoder.Decode(&x) // fatal error: runtime: out of memory
fmt.Println("Decoding finished.")
}
```
We can write less code by using struct tag options:
- `toarray`: encode without field names (decode back to original struct)
- `keyasint`: encode field names as integers (decode back to original struct)
- `omitempty`: omit empty field when encoding
- `omitzero`: omit zero-value field when encoding
<hr/>
As a special case, struct field tag "-" omits the field.
</details>
`fxamacker/cbor` is fast at rejecting malformed CBOR data. E.g. attempts to
decode 10 bytes of malicious CBOR data to `[]byte` (with default settings):
| Codec | Speed (ns/op) | Memory | Allocs |
| :---- | ------------: | -----: | -----: |
| fxamacker/cbor 2.5.0 | 44 ± 5% | 32 B/op | 2 allocs/op |
| ugorji/go 1.2.11 | 5353261 ± 4% | 67111321 B/op | 13 allocs/op |
<details><summary>Benchmark details</summary><p/>
Latest comparison used:
- Input: `[]byte{0x9B, 0x00, 0x00, 0x42, 0xFA, 0x42, 0xFA, 0x42, 0xFA, 0x42}`
- go1.19.10, linux/amd64, i5-13600K (disabled all e-cores, DDR4 @2933)
- go test -bench=. -benchmem -count=20
#### Prior comparisons
| Codec | Speed (ns/op) | Memory | Allocs |
| :---- | ------------: | -----: | -----: |
| fxamacker/cbor 2.5.0-beta2 | 44.33 ± 2% | 32 B/op | 2 allocs/op |
| fxamacker/cbor 0.1.0 - 2.4.0 | ~44.68 ± 6% | 32 B/op | 2 allocs/op |
| ugorji/go 1.2.10 | 5524792.50 ± 3% | 67110491 B/op | 12 allocs/op |
| ugorji/go 1.1.0 - 1.2.6 | 💥 runtime: | out of memory: | cannot allocate |
- Input: `[]byte{0x9B, 0x00, 0x00, 0x42, 0xFA, 0x42, 0xFA, 0x42, 0xFA, 0x42}`
- go1.19.6, linux/amd64, i5-13600K (DDR4)
- go test -bench=. -benchmem -count=20
<hr/>
</details>
### Smaller Encodings with Struct Tags
Struct tags (`toarray`, `keyasint`, `omitempty`) reduce encoded size of structs.
<details><summary>Example encoding 3-level nested Go struct to 1 byte CBOR</summary><p/>
https://go.dev/play/p/YxwvfPdFQG2
```Go
// Example encoding nested struct (with omitempty tag)
// - encoding/json: 18 byte JSON
// - fxamacker/cbor: 1 byte CBOR
package main
import (
"encoding/hex"
"encoding/json"
"fmt"
"github.com/fxamacker/cbor/v2"
)
type GrandChild struct {
Quux int `json:",omitempty"`
}
type Child struct {
Baz int `json:",omitempty"`
Qux GrandChild `json:",omitempty"`
}
type Parent struct {
Foo Child `json:",omitempty"`
Bar int `json:",omitempty"`
}
func cb() {
results, _ := cbor.Marshal(Parent{})
fmt.Println("hex(CBOR): " + hex.EncodeToString(results))
text, _ := cbor.Diagnose(results) // Diagnostic Notation
fmt.Println("DN: " + text)
}
func js() {
results, _ := json.Marshal(Parent{})
fmt.Println("hex(JSON): " + hex.EncodeToString(results))
text := string(results) // JSON
fmt.Println("JSON: " + text)
}
func main() {
cb()
fmt.Println("-------------")
js()
}
```
Output (DN is Diagnostic Notation):
```
hex(CBOR): a0
DN: {}
-------------
hex(JSON): 7b22466f6f223a7b22517578223a7b7d7d7d
JSON: {"Foo":{"Qux":{}}}
```
<hr/>
</details>
Example using different struct tags together:
NOTE: When a struct uses `toarray`, the encoder will ignore `omitempty` and `omitzero` to prevent position of encoded array elements from changing. This allows decoder to match encoded elements to their Go struct field.
![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/cbor_struct_tags_api.svg?sanitize=1 "CBOR API and Go Struct Tags")
API is mostly same as `encoding/json`, plus interfaces that simplify concurrency for CBOR options.
> [!NOTE]
> `fxamacker/cbor` can encode a 3-level nested Go struct to 1 byte!
> - `encoding/json`: 18 bytes of JSON
> - `fxamacker/cbor`: 1 byte of CBOR
>
> <details><summary> 🔎&nbsp; Encoding 3-level nested Go struct with omitempty</summary><p/>
>
> https://go.dev/play/p/YxwvfPdFQG2
>
> ```Go
> // Example encoding nested struct (with omitempty tag)
> // - encoding/json: 18 byte JSON
> // - fxamacker/cbor: 1 byte CBOR
>
> package main
>
> import (
> "encoding/hex"
> "encoding/json"
> "fmt"
>
> "github.com/fxamacker/cbor/v2"
> )
>
> type GrandChild struct {
> Quux int `json:",omitempty"`
> }
>
> type Child struct {
> Baz int `json:",omitempty"`
> Qux GrandChild `json:",omitempty"`
> }
>
> type Parent struct {
> Foo Child `json:",omitempty"`
> Bar int `json:",omitempty"`
> }
>
> func cb() {
> results, _ := cbor.Marshal(Parent{})
> fmt.Println("hex(CBOR): " + hex.EncodeToString(results))
>
> text, _ := cbor.Diagnose(results) // Diagnostic Notation
> fmt.Println("DN: " + text)
> }
>
> func js() {
> results, _ := json.Marshal(Parent{})
> fmt.Println("hex(JSON): " + hex.EncodeToString(results))
>
> text := string(results) // JSON
> fmt.Println("JSON: " + text)
> }
>
> func main() {
> cb()
> fmt.Println("-------------")
> js()
> }
> ```
>
> Output (DN is Diagnostic Notation):
> ```
> hex(CBOR): a0
> DN: {}
> -------------
> hex(JSON): 7b22466f6f223a7b22517578223a7b7d7d7d
> JSON: {"Foo":{"Qux":{}}}
> ```
>
> </details>
## Quick Start
__Install__: `go get github.com/fxamacker/cbor/v2` and `import "github.com/fxamacker/cbor/v2"`.
> [!TIP]
>
> Tinygo users can try beta/experimental branch [feature/cbor-tinygo-beta](https://github.com/fxamacker/cbor/tree/feature/cbor-tinygo-beta).
>
> <details><summary> 🔎&nbsp; More about tinygo feature branch</summary>
>
> ### Tinygo
>
> Branch [feature/cbor-tinygo-beta](https://github.com/fxamacker/cbor/tree/feature/cbor-tinygo-beta) is based on fxamacker/cbor v2.7.0 and it can be compiled using tinygo v0.33 (also compiles with golang/go).
>
> It passes unit tests (with both go1.22 and tinygo v0.33) and is considered beta/experimental for tinygo.
>
> :warning: The `feature/cbor-tinygo-beta` branch does not get fuzz tested yet.
>
> Changes in this feature branch only affect tinygo compiled software. Summary of changes:
> - default `DecOptions.MaxNestedLevels` is reduced to 16 (was 32). User can specify higher limit but 24+ crashes tests when compiled with tinygo v0.33.
> - disabled decoding CBOR tag data to Go interface because tinygo v0.33 is missing needed feature.
> - encoding error message can be different when encoding function type.
>
> Related tinygo issues:
> - https://github.com/tinygo-org/tinygo/issues/4277
> - https://github.com/tinygo-org/tinygo/issues/4458
>
> </details>
### Key Points
This library can encode and decode CBOR (RFC 8949) and CBOR Sequences (RFC 8742).
@@ -252,16 +294,17 @@ rest, err = cbor.UnmarshalFirst(b, &v) // decode []byte b to v
// DiagnoseFirst translates first CBOR data item to text and returns remaining bytes.
text, rest, err = cbor.DiagnoseFirst(b) // decode []byte b to Diagnostic Notation text
// NOTE: Unmarshal returns ExtraneousDataError if there are remaining bytes,
// but new funcs UnmarshalFirst and DiagnoseFirst do not.
// NOTE: Unmarshal() returns ExtraneousDataError if there are remaining bytes, but
// UnmarshalFirst() and DiagnoseFirst() allow trailing bytes.
```
__IMPORTANT__: 👉 CBOR settings allow trade-offs between speed, security, encoding size, etc.
- Different CBOR libraries may use different default settings.
- CBOR-based formats or protocols usually require specific settings.
For example, WebAuthn uses "CTAP2 Canonical CBOR" which is available as a preset.
> [!IMPORTANT]
> CBOR settings allow trade-offs between speed, security, encoding size, etc.
>
> - Different CBOR libraries may use different default settings.
> - CBOR-based formats or protocols usually require specific settings.
>
> For example, WebAuthn uses "CTAP2 Canonical CBOR" which is available as a preset.
### Presets
@@ -312,9 +355,63 @@ err = em.MarshalToBuffer(v, &buf) // encode v to provided buf
### Struct Tags
Struct tags (`toarray`, `keyasint`, `omitempty`) reduce encoded size of structs.
Struct tag options (`toarray`, `keyasint`, `omitempty`, `omitzero`) reduce encoded size of structs.
<details><summary>Example encoding 3-level nested Go struct to 1 byte CBOR</summary><p/>
As a special case, struct field tag "-" omits the field.
<details><summary> 🔎&nbsp; Example encoding with struct field tag "-"</summary><p/>
https://go.dev/play/p/aWEIFxd7InX
```Go
// https://github.com/fxamacker/cbor/issues/652
package main
import (
"encoding/json"
"fmt"
"github.com/fxamacker/cbor/v2"
)
// The `cbor:"-"` tag omits the Type field when encoding to CBOR.
type Entity struct {
_ struct{} `cbor:",toarray"`
ID uint64 `json:"id"`
Type string `cbor:"-" json:"typeOf"`
Name string `json:"name"`
}
func main() {
entity := Entity{
ID: 1,
Type: "int64",
Name: "Identifier",
}
c, _ := cbor.Marshal(entity)
diag, _ := cbor.Diagnose(c)
fmt.Printf("CBOR in hex: %x\n", c)
fmt.Printf("CBOR in edn: %s\n", diag)
j, _ := json.Marshal(entity)
fmt.Printf("JSON: %s\n", string(j))
fmt.Printf("JSON encoding is %d bytes\n", len(j))
fmt.Printf("CBOR encoding is %d bytes\n", len(c))
// Output:
// CBOR in hex: 82016a4964656e746966696572
// CBOR in edn: [1, "Identifier"]
// JSON: {"id":1,"typeOf":"int64","name":"Identifier"}
// JSON encoding is 45 bytes
// CBOR encoding is 13 bytes
}
```
</details>
<details><summary> 🔎&nbsp; Example encoding 3-level nested Go struct to 1 byte CBOR</summary><p/>
https://go.dev/play/p/YxwvfPdFQG2
@@ -382,13 +479,13 @@ JSON: {"Foo":{"Qux":{}}}
</details>
<details><summary>Example using several struct tags</summary><p/>
<details><summary> 🔎&nbsp; Example using struct tag options</summary><p/>
![alt text](https://github.com/fxamacker/images/raw/master/cbor/v2.3.0/cbor_struct_tags_api.svg?sanitize=1 "CBOR API and Go Struct Tags")
</details>
Struct tags simplify use of CBOR-based protocols that require CBOR arrays or maps with integer keys.
Struct tag options simplify use of CBOR-based protocols that require CBOR arrays or maps with integer keys.
### CBOR Tags
@@ -404,7 +501,7 @@ em, err := opts.EncModeWithSharedTags(ts) // mutable shared CBOR tags
`TagSet` and modes using it are safe for concurrent use. Equivalent API is available for `DecMode`.
<details><summary>Example using TagSet and TagOptions</summary><p/>
<details><summary> 🔎&nbsp; Example using TagSet and TagOptions</summary><p/>
```go
// Use signedCWT struct defined in "Decoding CWT" example.
@@ -430,16 +527,149 @@ if err := dm.Unmarshal(data, &v); err != nil {
em, _ := cbor.EncOptions{}.EncModeWithTags(tags)
// Marshal signedCWT with tag number.
if data, err := cbor.Marshal(v); err != nil {
if data, err := em.Marshal(v); err != nil {
return err
}
```
</details>
👉 `fxamacker/cbor` allows user apps to use almost any current or future CBOR tag number by implementing `cbor.Marshaler` and `cbor.Unmarshaler` interfaces.
Basically, `MarshalCBOR` and `UnmarshalCBOR` functions can be implemented by user apps and those functions will automatically be called by this CBOR codec's `Marshal`, `Unmarshal`, etc.
The following [example](https://github.com/fxamacker/cbor/blob/master/example_embedded_json_tag_for_cbor_test.go) shows how to encode and decode a tagged CBOR data item with tag number 262. The tag content is a JSON object "embedded" as a CBOR byte string (major type 2).
<details><summary> 🔎&nbsp; Example using Embedded JSON Tag for CBOR (tag 262)</summary>
```go
// https://github.com/fxamacker/cbor/issues/657
package cbor_test
// NOTE: RFC 8949 does not mention tag number 262. IANA assigned
// CBOR tag number 262 as "Embedded JSON Object" specified by the
// document Embedded JSON Tag for CBOR:
//
// "Tag 262 can be applied to a byte string (major type 2) to indicate
// that the byte string is a JSON Object. The length of the byte string
// indicates the content."
//
// For more info, see Embedded JSON Tag for CBOR at:
// https://github.com/toravir/CBOR-Tag-Specs/blob/master/embeddedJSON.md
import (
"bytes"
"encoding/json"
"fmt"
"github.com/fxamacker/cbor/v2"
)
// cborTagNumForEmbeddedJSON is the CBOR tag number 262.
const cborTagNumForEmbeddedJSON = 262
// EmbeddedJSON represents a Go value to be encoded as a tagged CBOR data item
// with tag number 262 and the tag content is a JSON object "embedded" as a
// CBOR byte string (major type 2).
type EmbeddedJSON struct {
any
}
func NewEmbeddedJSON(val any) EmbeddedJSON {
return EmbeddedJSON{val}
}
// MarshalCBOR encodes EmbeddedJSON to a tagged CBOR data item with the
// tag number 262 and the tag content is a JSON object that is
// "embedded" as a CBOR byte string.
func (v EmbeddedJSON) MarshalCBOR() ([]byte, error) {
// Encode v to JSON object.
data, err := json.Marshal(v)
if err != nil {
return nil, err
}
// Create cbor.Tag representing a tagged CBOR data item.
tag := cbor.Tag{
Number: cborTagNumForEmbeddedJSON,
Content: data,
}
// Marshal to a tagged CBOR data item.
return cbor.Marshal(tag)
}
// UnmarshalCBOR decodes a tagged CBOR data item to EmbeddedJSON.
// The byte slice provided to this function must contain a single
// tagged CBOR data item with the tag number 262 and tag content
// must be a JSON object "embedded" as a CBOR byte string.
func (v *EmbeddedJSON) UnmarshalCBOR(b []byte) error {
// Unmarshal tagged CBOR data item.
var tag cbor.Tag
if err := cbor.Unmarshal(b, &tag); err != nil {
return err
}
// Check tag number.
if tag.Number != cborTagNumForEmbeddedJSON {
return fmt.Errorf("got tag number %d, expect tag number %d", tag.Number, cborTagNumForEmbeddedJSON)
}
// Check tag content.
jsonData, isByteString := tag.Content.([]byte)
if !isByteString {
return fmt.Errorf("got tag content type %T, expect tag content []byte", tag.Content)
}
// Unmarshal JSON object.
return json.Unmarshal(jsonData, v)
}
// MarshalJSON encodes EmbeddedJSON to a JSON object.
func (v EmbeddedJSON) MarshalJSON() ([]byte, error) {
return json.Marshal(v.any)
}
// UnmarshalJSON decodes a JSON object.
func (v *EmbeddedJSON) UnmarshalJSON(b []byte) error {
dec := json.NewDecoder(bytes.NewReader(b))
dec.UseNumber()
return dec.Decode(&v.any)
}
func Example_embeddedJSONTagForCBOR() {
value := NewEmbeddedJSON(map[string]any{
"name": "gopher",
"id": json.Number("42"),
})
data, err := cbor.Marshal(value)
if err != nil {
panic(err)
}
fmt.Printf("cbor: %x\n", data)
var v EmbeddedJSON
err = cbor.Unmarshal(data, &v)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", v.any)
for k, v := range v.any.(map[string]any) {
fmt.Printf(" %s: %v (%T)\n", k, v, v)
}
}
```
</details>
### Functions and Interfaces
<details><summary>Functions and interfaces at a glance</summary><p/>
<details><summary> 🔎&nbsp; Functions and interfaces at a glance</summary><p/>
Common functions with same API as `encoding/json`:
- `Marshal`, `Unmarshal`
@@ -453,7 +683,7 @@ because RFC 8949 treats CBOR data item with remaining bytes as malformed.
Other useful functions:
- `Diagnose`, `DiagnoseFirst` produce human-readable [Extended Diagnostic Notation](https://www.rfc-editor.org/rfc/rfc8610.html#appendix-G) from CBOR data.
- `UnmarshalFirst` decodes first CBOR data item and return any remaining bytes.
- `Wellformed` returns true if the the CBOR data item is well-formed.
- `Wellformed` returns true if the CBOR data item is well-formed.
Interfaces identical or comparable to Go `encoding` packages include:
`Marshaler`, `Unmarshaler`, `BinaryMarshaler`, and `BinaryUnmarshaler`.
@@ -472,15 +702,28 @@ Default limits may need to be increased for systems handling very large data (e.
## Status
v2.7.0 (June 23, 2024) adds features and improvements that help large projects (e.g. Kubernetes) use CBOR as an alternative to JSON and Protocol Buffers. Other improvements include speedups, improved memory use, bug fixes, new serialization options, etc. It passed fuzz tests (5+ billion executions) and is production quality.
[v2.9.0](https://github.com/fxamacker/cbor/releases/tag/v2.9.0) (Jul 13, 2025) improved interoperability/transcoding between CBOR & JSON, refactored tests, and improved docs.
- Add opt-in support for `encoding.TextMarshaler` and `encoding.TextUnmarshaler` to encode and decode from CBOR text string.
- Add opt-in support for `json.Marshaler` and `json.Unmarshaler` via user-provided transcoding function.
- Update docs for TimeMode, Tag, RawTag, and add example for Embedded JSON Tag for CBOR.
v2.9.0 passed fuzz tests and is production quality.
The minimum version of Go required to build:
- v2.8.0 and newer releases require go 1.20+.
- v2.7.1 and older releases require go 1.17+.
For more details, see [release notes](https://github.com/fxamacker/cbor/releases).
### Prior Release
### Prior Releases
[v2.8.0](https://github.com/fxamacker/cbor/releases/tag/v2.8.0) (March 30, 2025) is a small release primarily to add `omitzero` option to struct field tags and fix bugs. It passed fuzz tests (billions of executions) and is production quality.
[v2.7.0](https://github.com/fxamacker/cbor/releases/tag/v2.7.0) (June 23, 2024) adds features and improvements that help large projects (e.g. Kubernetes) use CBOR as an alternative to JSON and Protocol Buffers. Other improvements include speedups, improved memory use, bug fixes, new serialization options, etc. It passed fuzz tests (5+ billion executions) and is production quality.
[v2.6.0](https://github.com/fxamacker/cbor/releases/tag/v2.6.0) (February 2024) adds important new features, optimizations, and bug fixes. It is especially useful to systems that need to convert data between CBOR and JSON. New options and optimizations improve handling of bignum, integers, maps, and strings.
v2.5.0 was released on Sunday, August 13, 2023 with new features and important bug fixes. It is fuzz tested and production quality after extended beta [v2.5.0-beta](https://github.com/fxamacker/cbor/releases/tag/v2.5.0-beta) (Dec 2022) -> [v2.5.0](https://github.com/fxamacker/cbor/releases/tag/v2.5.0) (Aug 2023).
[v2.5.0](https://github.com/fxamacker/cbor/releases/tag/v2.5.0) was released on Sunday, August 13, 2023 with new features and important bug fixes. It is fuzz tested and production quality after extended beta [v2.5.0-beta](https://github.com/fxamacker/cbor/releases/tag/v2.5.0-beta) (Dec 2022) -> [v2.5.0](https://github.com/fxamacker/cbor/releases/tag/v2.5.0) (Aug 2023).
__IMPORTANT__: 👉 Before upgrading from v2.4 or older release, please read the notable changes highlighted in the release notes. v2.5.0 is a large release with bug fixes to error handling for extraneous data in `Unmarshal`, etc. that should be reviewed before upgrading.
@@ -489,7 +732,7 @@ See [v2.5.0 release notes](https://github.com/fxamacker/cbor/releases/tag/v2.5.0
See ["Version and API Changes"](https://github.com/fxamacker/cbor#versions-and-api-changes) section for more info about version numbering, etc.
<!--
<details><summary>👉 Benchmark Comparison: v2.4.0 vs v2.5.0</summary><p/>
<details><summary> 🔎&nbsp; Benchmark Comparison: v2.4.0 vs v2.5.0</summary><p/>
TODO: Update to v2.4.0 vs 2.5.0 (not beta2).
@@ -549,9 +792,9 @@ geomean 2.782
## Who uses fxamacker/cbor
`fxamacker/cbor` is used in projects by Arm Ltd., Berlin Institute of Health at Charité, Chainlink, Cisco, Confidential Computing Consortium, ConsenSys, Dapper&nbsp;Labs, EdgeX&nbsp;Foundry, F5, FIDO Alliance, Fraunhofer&#8209;AISEC, Kubernetes, Let's Encrypt (ISRG), Linux&nbsp;Foundation, Matrix.org, Microsoft, Mozilla, National&nbsp;Cybersecurity&nbsp;Agency&nbsp;of&nbsp;France (govt), Netherlands (govt), Oasis Protocol, Smallstep, Tailscale, Taurus SA, Teleport, TIBCO, and others.
`fxamacker/cbor` is used in projects by Arm Ltd., Berlin Institute of Health at Charité, Chainlink, Confidential&nbsp;Computing&nbsp;Consortium, ConsenSys, EdgeX&nbsp;Foundry, F5, Flow&nbsp;Foundation, Fraunhofer&#8209;AISEC, IBM, Kubernetes, Let's&nbsp;Encrypt&nbsp;(ISRG), Linaro, Linux&nbsp;Foundation, Matrix.org, Microsoft, National&nbsp;Cybersecurity&nbsp;Agency&nbsp;of&nbsp;France&nbsp;(govt), Netherlands&nbsp;(govt), Oasis&nbsp;Protocol, Red Hat OpenShift, Smallstep, Tailscale, Taurus SA, TIBCO, Veraison, and others.
`fxamacker/cbor` passed multiple confidential security assessments. A [nonconfidential security assessment](https://github.com/veraison/go-cose/blob/v1.0.0-rc.1/reports/NCC_Microsoft-go-cose-Report_2022-05-26_v1.0.pdf) (prepared by NCC Group for Microsoft Corporation) includes a subset of fxamacker/cbor v2.4.0 in its scope.
`fxamacker/cbor` passed multiple confidential security assessments in 2022. A [nonconfidential security assessment](https://github.com/veraison/go-cose/blob/v1.0.0-rc.1/reports/NCC_Microsoft-go-cose-Report_2022-05-26_v1.0.pdf) (prepared by NCC Group for Microsoft Corporation) assessed a subset of fxamacker/cbor v2.4.
## Standards
@@ -588,7 +831,7 @@ By default, decoder treats time values of floating-point NaN and Infinity as if
__Click to expand topic:__
<details>
<summary>Duplicate Map Keys</summary><p>
<summary> 🔎&nbsp; Duplicate Map Keys</summary><p>
This library provides options for fast detection and rejection of duplicate map keys based on applying a Go-specific data model to CBOR's extended generic data model in order to determine duplicate vs distinct map keys. Detection relies on whether the CBOR map key would be a duplicate "key" when decoded and applied to the user-provided Go map or struct.
@@ -601,7 +844,7 @@ APF suffix means "Allow Partial Fill" so the destination map or struct can conta
</details>
<details>
<summary>Tag Validity</summary><p>
<summary> 🔎&nbsp; Tag Validity</summary><p>
This library checks tag validity for built-in tags (currently tag numbers 0, 1, 2, 3, and 55799):

View File

@@ -38,11 +38,38 @@ func (bs ByteString) MarshalCBOR() ([]byte, error) {
// UnmarshalCBOR decodes CBOR byte string (major type 2) to ByteString.
// Decoding CBOR null and CBOR undefined sets ByteString to be empty.
//
// Deprecated: No longer used by this codec; kept for compatibility
// with user apps that directly call this function.
func (bs *ByteString) UnmarshalCBOR(data []byte) error {
if bs == nil {
return errors.New("cbor.ByteString: UnmarshalCBOR on nil pointer")
}
d := decoder{data: data, dm: defaultDecMode}
// Check well-formedness of CBOR data item.
// ByteString.UnmarshalCBOR() is exported, so
// the codec needs to support same behavior for:
// - Unmarshal(data, *ByteString)
// - ByteString.UnmarshalCBOR(data)
err := d.wellformed(false, false)
if err != nil {
return err
}
return bs.unmarshalCBOR(data)
}
// unmarshalCBOR decodes CBOR byte string (major type 2) to ByteString.
// Decoding CBOR null and CBOR undefined sets ByteString to be empty.
// This function assumes data is well-formed, and does not perform bounds checking.
// This function is called by Unmarshal().
func (bs *ByteString) unmarshalCBOR(data []byte) error {
if bs == nil {
return errors.New("cbor.ByteString: UnmarshalCBOR on nil pointer")
}
// Decoding CBOR null and CBOR undefined to ByteString resets data.
// This behavior is similar to decoding CBOR null and CBOR undefined to []byte.
if len(data) == 1 && (data[0] == 0xf6 || data[0] == 0xf7) {

View File

@@ -17,6 +17,7 @@ import (
type encodeFuncs struct {
ef encodeFunc
ief isEmptyFunc
izf isZeroFunc
}
var (
@@ -31,10 +32,12 @@ type specialType int
const (
specialTypeNone specialType = iota
specialTypeUnmarshalerIface
specialTypeUnexportedUnmarshalerIface
specialTypeEmptyIface
specialTypeIface
specialTypeTag
specialTypeTime
specialTypeJSONUnmarshalerIface
)
type typeInfo struct {
@@ -50,7 +53,7 @@ type typeInfo struct {
func newTypeInfo(t reflect.Type) *typeInfo {
tInfo := typeInfo{typ: t, kind: t.Kind()}
for t.Kind() == reflect.Ptr {
for t.Kind() == reflect.Pointer {
t = t.Elem()
}
@@ -69,8 +72,12 @@ func newTypeInfo(t reflect.Type) *typeInfo {
tInfo.spclType = specialTypeTag
} else if t == typeTime {
tInfo.spclType = specialTypeTime
} else if reflect.PtrTo(t).Implements(typeUnmarshaler) {
} else if reflect.PointerTo(t).Implements(typeUnexportedUnmarshaler) {
tInfo.spclType = specialTypeUnexportedUnmarshalerIface
} else if reflect.PointerTo(t).Implements(typeUnmarshaler) {
tInfo.spclType = specialTypeUnmarshalerIface
} else if reflect.PointerTo(t).Implements(typeJSONUnmarshaler) {
tInfo.spclType = specialTypeJSONUnmarshalerIface
}
switch k {
@@ -237,7 +244,7 @@ func getEncodingStructType(t reflect.Type) (*encodingStructType, error) {
e := getEncodeBuffer()
for i := 0; i < len(flds); i++ {
// Get field's encodeFunc
flds[i].ef, flds[i].ief = getEncodeFunc(flds[i].typ)
flds[i].ef, flds[i].ief, flds[i].izf = getEncodeFunc(flds[i].typ)
if flds[i].ef == nil {
err = &UnsupportedTypeError{t}
break
@@ -321,7 +328,7 @@ func getEncodingStructType(t reflect.Type) (*encodingStructType, error) {
func getEncodingStructToArrayType(t reflect.Type, flds fields) (*encodingStructType, error) {
for i := 0; i < len(flds); i++ {
// Get field's encodeFunc
flds[i].ef, flds[i].ief = getEncodeFunc(flds[i].typ)
flds[i].ef, flds[i].ief, flds[i].izf = getEncodeFunc(flds[i].typ)
if flds[i].ef == nil {
structType := &encodingStructType{err: &UnsupportedTypeError{t}}
encodingStructTypeCache.Store(t, structType)
@@ -337,14 +344,14 @@ func getEncodingStructToArrayType(t reflect.Type, flds fields) (*encodingStructT
return structType, structType.err
}
func getEncodeFunc(t reflect.Type) (encodeFunc, isEmptyFunc) {
func getEncodeFunc(t reflect.Type) (encodeFunc, isEmptyFunc, isZeroFunc) {
if v, _ := encodeFuncCache.Load(t); v != nil {
fs := v.(encodeFuncs)
return fs.ef, fs.ief
return fs.ef, fs.ief, fs.izf
}
ef, ief := getEncodeFuncInternal(t)
encodeFuncCache.Store(t, encodeFuncs{ef, ief})
return ef, ief
ef, ief, izf := getEncodeFuncInternal(t)
encodeFuncCache.Store(t, encodeFuncs{ef, ief, izf})
return ef, ief, izf
}
func getTypeInfo(t reflect.Type) *typeInfo {

View File

@@ -5,6 +5,7 @@ package cbor
import (
"fmt"
"io"
"strconv"
)
@@ -180,3 +181,11 @@ func validBuiltinTag(tagNum uint64, contentHead byte) error {
return nil
}
// Transcoder is a scheme for transcoding a single CBOR encoded data item to or from a different
// data format.
type Transcoder interface {
// Transcode reads the data item in its source format from a Reader and writes a
// corresponding representation in its destination format to a Writer.
Transcode(dst io.Writer, src io.Reader) error
}

View File

@@ -4,6 +4,7 @@
package cbor
import (
"bytes"
"encoding"
"encoding/base64"
"encoding/binary"
@@ -94,7 +95,7 @@ import (
//
// To unmarshal CBOR null (0xf6) and undefined (0xf7) values into a
// slice/map/pointer, Unmarshal sets Go value to nil. Because null is often
// used to mean "not present", unmarshalling CBOR null and undefined value
// used to mean "not present", unmarshaling CBOR null and undefined value
// into any other Go type has no effect and returns no error.
//
// Unmarshal supports CBOR tag 55799 (self-describe CBOR), tag 0 and 1 (time),
@@ -104,7 +105,7 @@ import (
// if there are any remaining bytes following the first valid CBOR data item.
// See UnmarshalFirst, if you want to unmarshal only the first
// CBOR data item without ExtraneousDataError caused by remaining bytes.
func Unmarshal(data []byte, v interface{}) error {
func Unmarshal(data []byte, v any) error {
return defaultDecMode.Unmarshal(data, v)
}
@@ -114,7 +115,7 @@ func Unmarshal(data []byte, v interface{}) error {
// If v is nil, not a pointer, or a nil pointer, UnmarshalFirst returns an error.
//
// See the documentation for Unmarshal for details.
func UnmarshalFirst(data []byte, v interface{}) (rest []byte, err error) {
func UnmarshalFirst(data []byte, v any) (rest []byte, err error) {
return defaultDecMode.UnmarshalFirst(data, v)
}
@@ -151,6 +152,10 @@ type Unmarshaler interface {
UnmarshalCBOR([]byte) error
}
type unmarshaler interface {
unmarshalCBOR([]byte) error
}
// InvalidUnmarshalError describes an invalid argument passed to Unmarshal.
type InvalidUnmarshalError struct {
s string
@@ -193,12 +198,12 @@ func (e *InvalidMapKeyTypeError) Error() string {
// DupMapKeyError describes detected duplicate map key in CBOR map.
type DupMapKeyError struct {
Key interface{}
Key any
Index int
}
func (e *DupMapKeyError) Error() string {
return fmt.Sprintf("cbor: found duplicate map key \"%v\" at map element index %d", e.Key, e.Index)
return fmt.Sprintf("cbor: found duplicate map key %#v at map element index %d", e.Key, e.Index)
}
// UnknownFieldError describes detected unknown field in CBOR map when decoding to Go struct.
@@ -383,7 +388,7 @@ const (
// - return UnmarshalTypeError if value doesn't fit into int64
IntDecConvertSignedOrFail
// IntDecConvertSigned affects how CBOR integers (major type 0 and 1) decode to Go interface{}.
// IntDecConvertSignedOrBigInt affects how CBOR integers (major type 0 and 1) decode to Go interface{}.
// It makes CBOR integers (major type 0 and 1) decode to:
// - int64 if value fits
// - big.Int or *big.Int (see BigIntDecMode) if value doesn't fit into int64
@@ -489,11 +494,11 @@ type BigIntDecMode int
const (
// BigIntDecodeValue makes CBOR bignum decode to big.Int (instead of *big.Int)
// when unmarshalling into a Go interface{}.
// when unmarshaling into a Go interface{}.
BigIntDecodeValue BigIntDecMode = iota
// BigIntDecodePointer makes CBOR bignum decode to *big.Int when
// unmarshalling into a Go interface{}.
// unmarshaling into a Go interface{}.
BigIntDecodePointer
maxBigIntDecMode
@@ -745,6 +750,25 @@ func (bum BinaryUnmarshalerMode) valid() bool {
return bum >= 0 && bum < maxBinaryUnmarshalerMode
}
// TextUnmarshalerMode specifies how to decode into types that implement
// encoding.TextUnmarshaler.
type TextUnmarshalerMode int
const (
// TextUnmarshalerNone does not recognize TextUnmarshaler implementations during decode.
TextUnmarshalerNone TextUnmarshalerMode = iota
// TextUnmarshalerTextString will invoke UnmarshalText on the contents of a CBOR text
// string when decoding into a value that implements TextUnmarshaler.
TextUnmarshalerTextString
maxTextUnmarshalerMode
)
func (tum TextUnmarshalerMode) valid() bool {
return tum >= 0 && tum < maxTextUnmarshalerMode
}
// DecOptions specifies decoding options.
type DecOptions struct {
// DupMapKey specifies whether to enforce duplicate map key.
@@ -793,7 +817,7 @@ type DecOptions struct {
// TagsMd specifies whether to allow CBOR tags (major type 6).
TagsMd TagsMode
// IntDec specifies which Go integer type (int64 or uint64) to use
// IntDec specifies which Go integer type (int64, uint64, or [big.Int]) to use
// when decoding CBOR int (major type 0 and 1) to Go interface{}.
IntDec IntDecMode
@@ -807,7 +831,7 @@ type DecOptions struct {
ExtraReturnErrors ExtraDecErrorCond
// DefaultMapType specifies Go map type to create and decode to
// when unmarshalling CBOR into an empty interface value.
// when unmarshaling CBOR into an empty interface value.
// By default, unmarshal uses map[interface{}]interface{}.
DefaultMapType reflect.Type
@@ -879,6 +903,15 @@ type DecOptions struct {
// BinaryUnmarshaler specifies how to decode into types that implement
// encoding.BinaryUnmarshaler.
BinaryUnmarshaler BinaryUnmarshalerMode
// TextUnmarshaler specifies how to decode into types that implement
// encoding.TextUnmarshaler.
TextUnmarshaler TextUnmarshalerMode
// JSONUnmarshalerTranscoder sets the transcoding scheme used to unmarshal types that
// implement json.Unmarshaler but do not also implement cbor.Unmarshaler. If nil, decoding
// behavior is not influenced by whether or not a type implements json.Unmarshaler.
JSONUnmarshalerTranscoder Transcoder
}
// DecMode returns DecMode with immutable options and no tags (safe for concurrency).
@@ -1091,33 +1124,39 @@ func (opts DecOptions) decMode() (*decMode, error) { //nolint:gocritic // ignore
return nil, errors.New("cbor: invalid BinaryUnmarshaler " + strconv.Itoa(int(opts.BinaryUnmarshaler)))
}
if !opts.TextUnmarshaler.valid() {
return nil, errors.New("cbor: invalid TextUnmarshaler " + strconv.Itoa(int(opts.TextUnmarshaler)))
}
dm := decMode{
dupMapKey: opts.DupMapKey,
timeTag: opts.TimeTag,
maxNestedLevels: opts.MaxNestedLevels,
maxArrayElements: opts.MaxArrayElements,
maxMapPairs: opts.MaxMapPairs,
indefLength: opts.IndefLength,
tagsMd: opts.TagsMd,
intDec: opts.IntDec,
mapKeyByteString: opts.MapKeyByteString,
extraReturnErrors: opts.ExtraReturnErrors,
defaultMapType: opts.DefaultMapType,
utf8: opts.UTF8,
fieldNameMatching: opts.FieldNameMatching,
bigIntDec: opts.BigIntDec,
defaultByteStringType: opts.DefaultByteStringType,
byteStringToString: opts.ByteStringToString,
fieldNameByteString: opts.FieldNameByteString,
unrecognizedTagToAny: opts.UnrecognizedTagToAny,
timeTagToAny: opts.TimeTagToAny,
simpleValues: simpleValues,
nanDec: opts.NaN,
infDec: opts.Inf,
byteStringToTime: opts.ByteStringToTime,
byteStringExpectedFormat: opts.ByteStringExpectedFormat,
bignumTag: opts.BignumTag,
binaryUnmarshaler: opts.BinaryUnmarshaler,
dupMapKey: opts.DupMapKey,
timeTag: opts.TimeTag,
maxNestedLevels: opts.MaxNestedLevels,
maxArrayElements: opts.MaxArrayElements,
maxMapPairs: opts.MaxMapPairs,
indefLength: opts.IndefLength,
tagsMd: opts.TagsMd,
intDec: opts.IntDec,
mapKeyByteString: opts.MapKeyByteString,
extraReturnErrors: opts.ExtraReturnErrors,
defaultMapType: opts.DefaultMapType,
utf8: opts.UTF8,
fieldNameMatching: opts.FieldNameMatching,
bigIntDec: opts.BigIntDec,
defaultByteStringType: opts.DefaultByteStringType,
byteStringToString: opts.ByteStringToString,
fieldNameByteString: opts.FieldNameByteString,
unrecognizedTagToAny: opts.UnrecognizedTagToAny,
timeTagToAny: opts.TimeTagToAny,
simpleValues: simpleValues,
nanDec: opts.NaN,
infDec: opts.Inf,
byteStringToTime: opts.ByteStringToTime,
byteStringExpectedFormat: opts.ByteStringExpectedFormat,
bignumTag: opts.BignumTag,
binaryUnmarshaler: opts.BinaryUnmarshaler,
textUnmarshaler: opts.TextUnmarshaler,
jsonUnmarshalerTranscoder: opts.JSONUnmarshalerTranscoder,
}
return &dm, nil
@@ -1130,7 +1169,7 @@ type DecMode interface {
// Unmarshal returns an error.
//
// See the documentation for Unmarshal for details.
Unmarshal(data []byte, v interface{}) error
Unmarshal(data []byte, v any) error
// UnmarshalFirst parses the first CBOR data item into the value pointed to by v
// using the decoding mode. Any remaining bytes are returned in rest.
@@ -1138,7 +1177,7 @@ type DecMode interface {
// If v is nil, not a pointer, or a nil pointer, UnmarshalFirst returns an error.
//
// See the documentation for Unmarshal for details.
UnmarshalFirst(data []byte, v interface{}) (rest []byte, err error)
UnmarshalFirst(data []byte, v any) (rest []byte, err error)
// Valid checks whether data is a well-formed encoded CBOR data item and
// that it complies with configurable restrictions such as MaxNestedLevels,
@@ -1170,33 +1209,35 @@ type DecMode interface {
}
type decMode struct {
tags tagProvider
dupMapKey DupMapKeyMode
timeTag DecTagMode
maxNestedLevels int
maxArrayElements int
maxMapPairs int
indefLength IndefLengthMode
tagsMd TagsMode
intDec IntDecMode
mapKeyByteString MapKeyByteStringMode
extraReturnErrors ExtraDecErrorCond
defaultMapType reflect.Type
utf8 UTF8Mode
fieldNameMatching FieldNameMatchingMode
bigIntDec BigIntDecMode
defaultByteStringType reflect.Type
byteStringToString ByteStringToStringMode
fieldNameByteString FieldNameByteStringMode
unrecognizedTagToAny UnrecognizedTagToAnyMode
timeTagToAny TimeTagToAnyMode
simpleValues *SimpleValueRegistry
nanDec NaNMode
infDec InfMode
byteStringToTime ByteStringToTimeMode
byteStringExpectedFormat ByteStringExpectedFormatMode
bignumTag BignumTagMode
binaryUnmarshaler BinaryUnmarshalerMode
tags tagProvider
dupMapKey DupMapKeyMode
timeTag DecTagMode
maxNestedLevels int
maxArrayElements int
maxMapPairs int
indefLength IndefLengthMode
tagsMd TagsMode
intDec IntDecMode
mapKeyByteString MapKeyByteStringMode
extraReturnErrors ExtraDecErrorCond
defaultMapType reflect.Type
utf8 UTF8Mode
fieldNameMatching FieldNameMatchingMode
bigIntDec BigIntDecMode
defaultByteStringType reflect.Type
byteStringToString ByteStringToStringMode
fieldNameByteString FieldNameByteStringMode
unrecognizedTagToAny UnrecognizedTagToAnyMode
timeTagToAny TimeTagToAnyMode
simpleValues *SimpleValueRegistry
nanDec NaNMode
infDec InfMode
byteStringToTime ByteStringToTimeMode
byteStringExpectedFormat ByteStringExpectedFormatMode
bignumTag BignumTagMode
binaryUnmarshaler BinaryUnmarshalerMode
textUnmarshaler TextUnmarshalerMode
jsonUnmarshalerTranscoder Transcoder
}
var defaultDecMode, _ = DecOptions{}.decMode()
@@ -1211,32 +1252,34 @@ func (dm *decMode) DecOptions() DecOptions {
}
return DecOptions{
DupMapKey: dm.dupMapKey,
TimeTag: dm.timeTag,
MaxNestedLevels: dm.maxNestedLevels,
MaxArrayElements: dm.maxArrayElements,
MaxMapPairs: dm.maxMapPairs,
IndefLength: dm.indefLength,
TagsMd: dm.tagsMd,
IntDec: dm.intDec,
MapKeyByteString: dm.mapKeyByteString,
ExtraReturnErrors: dm.extraReturnErrors,
DefaultMapType: dm.defaultMapType,
UTF8: dm.utf8,
FieldNameMatching: dm.fieldNameMatching,
BigIntDec: dm.bigIntDec,
DefaultByteStringType: dm.defaultByteStringType,
ByteStringToString: dm.byteStringToString,
FieldNameByteString: dm.fieldNameByteString,
UnrecognizedTagToAny: dm.unrecognizedTagToAny,
TimeTagToAny: dm.timeTagToAny,
SimpleValues: simpleValues,
NaN: dm.nanDec,
Inf: dm.infDec,
ByteStringToTime: dm.byteStringToTime,
ByteStringExpectedFormat: dm.byteStringExpectedFormat,
BignumTag: dm.bignumTag,
BinaryUnmarshaler: dm.binaryUnmarshaler,
DupMapKey: dm.dupMapKey,
TimeTag: dm.timeTag,
MaxNestedLevels: dm.maxNestedLevels,
MaxArrayElements: dm.maxArrayElements,
MaxMapPairs: dm.maxMapPairs,
IndefLength: dm.indefLength,
TagsMd: dm.tagsMd,
IntDec: dm.intDec,
MapKeyByteString: dm.mapKeyByteString,
ExtraReturnErrors: dm.extraReturnErrors,
DefaultMapType: dm.defaultMapType,
UTF8: dm.utf8,
FieldNameMatching: dm.fieldNameMatching,
BigIntDec: dm.bigIntDec,
DefaultByteStringType: dm.defaultByteStringType,
ByteStringToString: dm.byteStringToString,
FieldNameByteString: dm.fieldNameByteString,
UnrecognizedTagToAny: dm.unrecognizedTagToAny,
TimeTagToAny: dm.timeTagToAny,
SimpleValues: simpleValues,
NaN: dm.nanDec,
Inf: dm.infDec,
ByteStringToTime: dm.byteStringToTime,
ByteStringExpectedFormat: dm.byteStringExpectedFormat,
BignumTag: dm.bignumTag,
BinaryUnmarshaler: dm.binaryUnmarshaler,
TextUnmarshaler: dm.textUnmarshaler,
JSONUnmarshalerTranscoder: dm.jsonUnmarshalerTranscoder,
}
}
@@ -1245,7 +1288,7 @@ func (dm *decMode) DecOptions() DecOptions {
// Unmarshal returns an error.
//
// See the documentation for Unmarshal for details.
func (dm *decMode) Unmarshal(data []byte, v interface{}) error {
func (dm *decMode) Unmarshal(data []byte, v any) error {
d := decoder{data: data, dm: dm}
// Check well-formedness.
@@ -1265,7 +1308,7 @@ func (dm *decMode) Unmarshal(data []byte, v interface{}) error {
// If v is nil, not a pointer, or a nil pointer, UnmarshalFirst returns an error.
//
// See the documentation for Unmarshal for details.
func (dm *decMode) UnmarshalFirst(data []byte, v interface{}) (rest []byte, err error) {
func (dm *decMode) UnmarshalFirst(data []byte, v any) (rest []byte, err error) {
d := decoder{data: data, dm: dm}
// check well-formedness.
@@ -1341,13 +1384,13 @@ type decoder struct {
// If CBOR data item fails to be decoded into v,
// error is returned and offset is moved to the next CBOR data item.
// Precondition: d.data contains at least one well-formed CBOR data item.
func (d *decoder) value(v interface{}) error {
func (d *decoder) value(v any) error {
// v can't be nil, non-pointer, or nil pointer value.
if v == nil {
return &InvalidUnmarshalError{"cbor: Unmarshal(nil)"}
}
rv := reflect.ValueOf(v)
if rv.Kind() != reflect.Ptr {
if rv.Kind() != reflect.Pointer {
return &InvalidUnmarshalError{"cbor: Unmarshal(non-pointer " + rv.Type().String() + ")"}
} else if rv.IsNil() {
return &InvalidUnmarshalError{"cbor: Unmarshal(nil " + rv.Type().String() + ")"}
@@ -1361,9 +1404,9 @@ func (d *decoder) value(v interface{}) error {
func (d *decoder) parseToValue(v reflect.Value, tInfo *typeInfo) error { //nolint:gocyclo
// Decode CBOR nil or CBOR undefined to pointer value by setting pointer value to nil.
if d.nextCBORNil() && v.Kind() == reflect.Ptr {
if d.nextCBORNil() && v.Kind() == reflect.Pointer {
d.skip()
v.Set(reflect.Zero(v.Type()))
v.SetZero()
return nil
}
@@ -1387,7 +1430,7 @@ func (d *decoder) parseToValue(v reflect.Value, tInfo *typeInfo) error { //nolin
registeredType := d.dm.tags.getTypeFromTagNum(tagNums)
if registeredType != nil {
if registeredType.Implements(tInfo.nonPtrType) ||
reflect.PtrTo(registeredType).Implements(tInfo.nonPtrType) {
reflect.PointerTo(registeredType).Implements(tInfo.nonPtrType) {
v.Set(reflect.New(registeredType))
v = v.Elem()
tInfo = getTypeInfo(registeredType)
@@ -1399,7 +1442,7 @@ func (d *decoder) parseToValue(v reflect.Value, tInfo *typeInfo) error { //nolin
// Create new value for the pointer v to point to.
// At this point, CBOR value is not nil/undefined if v is a pointer.
for v.Kind() == reflect.Ptr {
for v.Kind() == reflect.Pointer {
if v.IsNil() {
if !v.CanSet() {
d.skip()
@@ -1460,6 +1503,17 @@ func (d *decoder) parseToValue(v reflect.Value, tInfo *typeInfo) error { //nolin
case specialTypeUnmarshalerIface:
return d.parseToUnmarshaler(v)
case specialTypeUnexportedUnmarshalerIface:
return d.parseToUnexportedUnmarshaler(v)
case specialTypeJSONUnmarshalerIface:
// This special type implies that the type does not also implement
// cbor.Umarshaler.
if d.dm.jsonUnmarshalerTranscoder == nil {
break
}
return d.parseToJSONUnmarshaler(v)
}
}
@@ -1516,14 +1570,14 @@ func (d *decoder) parseToValue(v reflect.Value, tInfo *typeInfo) error { //nolin
return err
}
copied = copied || converted
return fillByteString(t, b, !copied, v, d.dm.byteStringToString, d.dm.binaryUnmarshaler)
return fillByteString(t, b, !copied, v, d.dm.byteStringToString, d.dm.binaryUnmarshaler, d.dm.textUnmarshaler)
case cborTypeTextString:
b, err := d.parseTextString()
if err != nil {
return err
}
return fillTextString(t, b, v)
return fillTextString(t, b, v, d.dm.textUnmarshaler)
case cborTypePrimitives:
_, ai, val := d.getHead()
@@ -1575,7 +1629,7 @@ func (d *decoder) parseToValue(v reflect.Value, tInfo *typeInfo) error { //nolin
return nil
}
if tInfo.nonPtrKind == reflect.Slice || tInfo.nonPtrKind == reflect.Array {
return fillByteString(t, b, !copied, v, ByteStringToStringForbidden, d.dm.binaryUnmarshaler)
return fillByteString(t, b, !copied, v, ByteStringToStringForbidden, d.dm.binaryUnmarshaler, d.dm.textUnmarshaler)
}
if bi.IsUint64() {
return fillPositiveInt(t, bi.Uint64(), v)
@@ -1598,7 +1652,7 @@ func (d *decoder) parseToValue(v reflect.Value, tInfo *typeInfo) error { //nolin
return nil
}
if tInfo.nonPtrKind == reflect.Slice || tInfo.nonPtrKind == reflect.Array {
return fillByteString(t, b, !copied, v, ByteStringToStringForbidden, d.dm.binaryUnmarshaler)
return fillByteString(t, b, !copied, v, ByteStringToStringForbidden, d.dm.binaryUnmarshaler, d.dm.textUnmarshaler)
}
if bi.IsInt64() {
return fillNegativeInt(t, bi.Int64(), v)
@@ -1788,12 +1842,12 @@ func (d *decoder) parseToTime() (time.Time, bool, error) {
// parseToUnmarshaler parses CBOR data to value implementing Unmarshaler interface.
// It assumes data is well-formed, and does not perform bounds checking.
func (d *decoder) parseToUnmarshaler(v reflect.Value) error {
if d.nextCBORNil() && v.Kind() == reflect.Ptr && v.IsNil() {
if d.nextCBORNil() && v.Kind() == reflect.Pointer && v.IsNil() {
d.skip()
return nil
}
if v.Kind() != reflect.Ptr && v.CanAddr() {
if v.Kind() != reflect.Pointer && v.CanAddr() {
v = v.Addr()
}
if u, ok := v.Interface().(Unmarshaler); ok {
@@ -1805,9 +1859,55 @@ func (d *decoder) parseToUnmarshaler(v reflect.Value) error {
return errors.New("cbor: failed to assert " + v.Type().String() + " as cbor.Unmarshaler")
}
// parseToUnexportedUnmarshaler parses CBOR data to value implementing unmarshaler interface.
// It assumes data is well-formed, and does not perform bounds checking.
func (d *decoder) parseToUnexportedUnmarshaler(v reflect.Value) error {
if d.nextCBORNil() && v.Kind() == reflect.Pointer && v.IsNil() {
d.skip()
return nil
}
if v.Kind() != reflect.Pointer && v.CanAddr() {
v = v.Addr()
}
if u, ok := v.Interface().(unmarshaler); ok {
start := d.off
d.skip()
return u.unmarshalCBOR(d.data[start:d.off])
}
d.skip()
return errors.New("cbor: failed to assert " + v.Type().String() + " as cbor.unmarshaler")
}
// parseToJSONUnmarshaler parses CBOR data to be transcoded to JSON and passed to the value's
// implementation of the json.Unmarshaler interface. It assumes data is well-formed, and does not
// perform bounds checking.
func (d *decoder) parseToJSONUnmarshaler(v reflect.Value) error {
if d.nextCBORNil() && v.Kind() == reflect.Pointer && v.IsNil() {
d.skip()
return nil
}
if v.Kind() != reflect.Pointer && v.CanAddr() {
v = v.Addr()
}
if u, ok := v.Interface().(jsonUnmarshaler); ok {
start := d.off
d.skip()
e := getEncodeBuffer()
defer putEncodeBuffer(e)
if err := d.dm.jsonUnmarshalerTranscoder.Transcode(e, bytes.NewReader(d.data[start:d.off])); err != nil {
return &TranscodeError{err: err, rtype: v.Type(), sourceFormat: "cbor", targetFormat: "json"}
}
return u.UnmarshalJSON(e.Bytes())
}
d.skip()
return errors.New("cbor: failed to assert " + v.Type().String() + " as json.Unmarshaler")
}
// parse parses CBOR data and returns value in default Go type.
// It assumes data is well-formed, and does not perform bounds checking.
func (d *decoder) parse(skipSelfDescribedTag bool) (interface{}, error) { //nolint:gocyclo
func (d *decoder) parse(skipSelfDescribedTag bool) (any, error) { //nolint:gocyclo
// Strip self-described CBOR tag number.
if skipSelfDescribedTag {
for d.nextCBORType() == cborTypeTag {
@@ -2224,15 +2324,15 @@ func (d *decoder) parseTextString() ([]byte, error) {
return b, nil
}
func (d *decoder) parseArray() ([]interface{}, error) {
func (d *decoder) parseArray() ([]any, error) {
_, _, val, indefiniteLength := d.getHeadWithIndefiniteLengthFlag()
hasSize := !indefiniteLength
count := int(val)
if !hasSize {
count = d.numOfItemsUntilBreak() // peek ahead to get array size to preallocate slice for better performance
}
v := make([]interface{}, count)
var e interface{}
v := make([]any, count)
var e any
var err, lastErr error
for i := 0; (hasSize && i < count) || (!hasSize && !d.foundBreak()); i++ {
if e, lastErr = d.parse(true); lastErr != nil {
@@ -2290,20 +2390,19 @@ func (d *decoder) parseArrayToArray(v reflect.Value, tInfo *typeInfo) error {
}
// Set remaining Go array elements to zero values.
if gi < vLen {
zeroV := reflect.Zero(tInfo.elemTypeInfo.typ)
for ; gi < vLen; gi++ {
v.Index(gi).Set(zeroV)
v.Index(gi).SetZero()
}
}
return err
}
func (d *decoder) parseMap() (interface{}, error) {
func (d *decoder) parseMap() (any, error) {
_, _, val, indefiniteLength := d.getHeadWithIndefiniteLengthFlag()
hasSize := !indefiniteLength
count := int(val)
m := make(map[interface{}]interface{})
var k, e interface{}
m := make(map[any]any)
var k, e any
var err, lastErr error
keyCount := 0
for i := 0; (hasSize && i < count) || (!hasSize && !d.foundBreak()); i++ {
@@ -2376,13 +2475,13 @@ func (d *decoder) parseMapToMap(v reflect.Value, tInfo *typeInfo) error { //noli
}
keyType, eleType := tInfo.keyTypeInfo.typ, tInfo.elemTypeInfo.typ
reuseKey, reuseEle := isImmutableKind(tInfo.keyTypeInfo.kind), isImmutableKind(tInfo.elemTypeInfo.kind)
var keyValue, eleValue, zeroKeyValue, zeroEleValue reflect.Value
var keyValue, eleValue reflect.Value
keyIsInterfaceType := keyType == typeIntf // If key type is interface{}, need to check if key value is hashable.
var err, lastErr error
keyCount := v.Len()
var existingKeys map[interface{}]bool // Store existing map keys, used for detecting duplicate map key.
var existingKeys map[any]bool // Store existing map keys, used for detecting duplicate map key.
if d.dm.dupMapKey == DupMapKeyEnforcedAPF {
existingKeys = make(map[interface{}]bool, keyCount)
existingKeys = make(map[any]bool, keyCount)
if keyCount > 0 {
vKeys := v.MapKeys()
for i := 0; i < len(vKeys); i++ {
@@ -2395,10 +2494,7 @@ func (d *decoder) parseMapToMap(v reflect.Value, tInfo *typeInfo) error { //noli
if !keyValue.IsValid() {
keyValue = reflect.New(keyType).Elem()
} else if !reuseKey {
if !zeroKeyValue.IsValid() {
zeroKeyValue = reflect.Zero(keyType)
}
keyValue.Set(zeroKeyValue)
keyValue.SetZero()
}
if lastErr = d.parseToValue(keyValue, tInfo.keyTypeInfo); lastErr != nil {
if err == nil {
@@ -2413,7 +2509,7 @@ func (d *decoder) parseMapToMap(v reflect.Value, tInfo *typeInfo) error { //noli
if !isHashableValue(keyValue.Elem()) {
var converted bool
if d.dm.mapKeyByteString == MapKeyByteStringAllowed {
var k interface{}
var k any
k, converted = convertByteSliceToByteString(keyValue.Elem().Interface())
if converted {
keyValue.Set(reflect.ValueOf(k))
@@ -2433,10 +2529,7 @@ func (d *decoder) parseMapToMap(v reflect.Value, tInfo *typeInfo) error { //noli
if !eleValue.IsValid() {
eleValue = reflect.New(eleType).Elem()
} else if !reuseEle {
if !zeroEleValue.IsValid() {
zeroEleValue = reflect.Zero(eleType)
}
eleValue.Set(zeroEleValue)
eleValue.SetZero()
}
if lastErr := d.parseToValue(eleValue, tInfo.elemTypeInfo); lastErr != nil {
if err == nil {
@@ -2584,7 +2677,7 @@ func (d *decoder) parseMapToStruct(v reflect.Value, tInfo *typeInfo) error { //n
// Keeps track of CBOR map keys to detect duplicate map key
keyCount := 0
var mapKeys map[interface{}]struct{}
var mapKeys map[any]struct{}
errOnUnknownField := (d.dm.extraReturnErrors & ExtraDecErrorUnknownField) > 0
@@ -2594,7 +2687,7 @@ MapEntryLoop:
// If duplicate field detection is enabled and the key at index j did not match any
// field, k will hold the map key.
var k interface{}
var k any
t := d.nextCBORType()
if t == cborTypeTextString || (t == cborTypeByteString && d.dm.fieldNameByteString == FieldNameByteStringAllowed) {
@@ -2764,7 +2857,7 @@ MapEntryLoop:
// check is never reached.
if d.dm.dupMapKey == DupMapKeyEnforcedAPF {
if mapKeys == nil {
mapKeys = make(map[interface{}]struct{}, 1)
mapKeys = make(map[any]struct{}, 1)
}
mapKeys[k] = struct{}{}
newKeyCount := len(mapKeys)
@@ -2968,20 +3061,25 @@ func (d *decoder) nextCBORNil() bool {
return d.data[d.off] == 0xf6 || d.data[d.off] == 0xf7
}
type jsonUnmarshaler interface{ UnmarshalJSON([]byte) error }
var (
typeIntf = reflect.TypeOf([]interface{}(nil)).Elem()
typeTime = reflect.TypeOf(time.Time{})
typeBigInt = reflect.TypeOf(big.Int{})
typeUnmarshaler = reflect.TypeOf((*Unmarshaler)(nil)).Elem()
typeBinaryUnmarshaler = reflect.TypeOf((*encoding.BinaryUnmarshaler)(nil)).Elem()
typeString = reflect.TypeOf("")
typeByteSlice = reflect.TypeOf([]byte(nil))
typeIntf = reflect.TypeOf([]any(nil)).Elem()
typeTime = reflect.TypeOf(time.Time{})
typeBigInt = reflect.TypeOf(big.Int{})
typeUnmarshaler = reflect.TypeOf((*Unmarshaler)(nil)).Elem()
typeUnexportedUnmarshaler = reflect.TypeOf((*unmarshaler)(nil)).Elem()
typeBinaryUnmarshaler = reflect.TypeOf((*encoding.BinaryUnmarshaler)(nil)).Elem()
typeTextUnmarshaler = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
typeJSONUnmarshaler = reflect.TypeOf((*jsonUnmarshaler)(nil)).Elem()
typeString = reflect.TypeOf("")
typeByteSlice = reflect.TypeOf([]byte(nil))
)
func fillNil(_ cborType, v reflect.Value) error {
switch v.Kind() {
case reflect.Slice, reflect.Map, reflect.Interface, reflect.Ptr:
v.Set(reflect.Zero(v.Type()))
case reflect.Slice, reflect.Map, reflect.Interface, reflect.Pointer:
v.SetZero()
return nil
}
return nil
@@ -3082,8 +3180,8 @@ func fillFloat(t cborType, val float64, v reflect.Value) error {
return &UnmarshalTypeError{CBORType: t.String(), GoType: v.Type().String()}
}
func fillByteString(t cborType, val []byte, shared bool, v reflect.Value, bsts ByteStringToStringMode, bum BinaryUnmarshalerMode) error {
if bum == BinaryUnmarshalerByteString && reflect.PtrTo(v.Type()).Implements(typeBinaryUnmarshaler) {
func fillByteString(t cborType, val []byte, shared bool, v reflect.Value, bsts ByteStringToStringMode, bum BinaryUnmarshalerMode, tum TextUnmarshalerMode) error {
if bum == BinaryUnmarshalerByteString && reflect.PointerTo(v.Type()).Implements(typeBinaryUnmarshaler) {
if v.CanAddr() {
v = v.Addr()
if u, ok := v.Interface().(encoding.BinaryUnmarshaler); ok {
@@ -3095,9 +3193,26 @@ func fillByteString(t cborType, val []byte, shared bool, v reflect.Value, bsts B
}
return errors.New("cbor: cannot set new value for " + v.Type().String())
}
if bsts != ByteStringToStringForbidden && v.Kind() == reflect.String {
v.SetString(string(val))
return nil
if bsts != ByteStringToStringForbidden {
if tum == TextUnmarshalerTextString && reflect.PointerTo(v.Type()).Implements(typeTextUnmarshaler) {
if v.CanAddr() {
v = v.Addr()
if u, ok := v.Interface().(encoding.TextUnmarshaler); ok {
// The contract of TextUnmarshaler forbids retaining the input
// bytes, so no copying is required even if val is shared.
if err := u.UnmarshalText(val); err != nil {
return fmt.Errorf("cbor: cannot unmarshal text for %s: %w", v.Type(), err)
}
return nil
}
}
return errors.New("cbor: cannot set new value for " + v.Type().String())
}
if v.Kind() == reflect.String {
v.SetString(string(val))
return nil
}
}
if v.Kind() == reflect.Slice && v.Type().Elem().Kind() == reflect.Uint8 {
src := val
@@ -3117,9 +3232,8 @@ func fillByteString(t cborType, val []byte, shared bool, v reflect.Value, bsts B
}
// Set remaining Go array elements to zero values.
if i < vLen {
zeroV := reflect.Zero(reflect.TypeOf(byte(0)))
for ; i < vLen; i++ {
v.Index(i).Set(zeroV)
v.Index(i).SetZero()
}
}
return nil
@@ -3127,11 +3241,28 @@ func fillByteString(t cborType, val []byte, shared bool, v reflect.Value, bsts B
return &UnmarshalTypeError{CBORType: t.String(), GoType: v.Type().String()}
}
func fillTextString(t cborType, val []byte, v reflect.Value) error {
func fillTextString(t cborType, val []byte, v reflect.Value, tum TextUnmarshalerMode) error {
// Check if the value implements TextUnmarshaler and the mode allows it
if tum == TextUnmarshalerTextString && reflect.PointerTo(v.Type()).Implements(typeTextUnmarshaler) {
if v.CanAddr() {
v = v.Addr()
if u, ok := v.Interface().(encoding.TextUnmarshaler); ok {
// The contract of TextUnmarshaler forbids retaining the input
// bytes, so no copying is required even if val is shared.
if err := u.UnmarshalText(val); err != nil {
return fmt.Errorf("cbor: cannot unmarshal text for %s: %w", v.Type(), err)
}
return nil
}
}
return errors.New("cbor: cannot set new value for " + v.Type().String())
}
if v.Kind() == reflect.String {
v.SetString(string(val))
return nil
}
return &UnmarshalTypeError{CBORType: t.String(), GoType: v.Type().String()}
}
@@ -3172,7 +3303,7 @@ func isHashableValue(rv reflect.Value) bool {
// This function also handles nested tags.
// CBOR data is already verified to be well-formed before this function is used,
// so the recursion won't exceed max nested levels.
func convertByteSliceToByteString(v interface{}) (interface{}, bool) {
func convertByteSliceToByteString(v any) (any, bool) {
switch v := v.(type) {
case []byte:
return ByteString(v), true

View File

@@ -2,15 +2,15 @@
// Licensed under the MIT License. See LICENSE in the project root for license information.
/*
Package cbor is a modern CBOR codec (RFC 8949 & RFC 7049) with CBOR tags,
Go struct tags (toarray/keyasint/omitempty), Core Deterministic Encoding,
Package cbor is a modern CBOR codec (RFC 8949 & RFC 8742) with CBOR tags,
Go struct tag options (toarray/keyasint/omitempty/omitzero), Core Deterministic Encoding,
CTAP2, Canonical CBOR, float64->32->16, and duplicate map key detection.
Encoding options allow "preferred serialization" by encoding integers and floats
to their smallest forms (e.g. float16) when values fit.
Struct tags like "keyasint", "toarray" and "omitempty" make CBOR data smaller
and easier to use with structs.
Struct tag options "keyasint", "toarray", "omitempty", and "omitzero" reduce encoding size
and reduce programming effort.
For example, "toarray" tag makes struct fields encode to CBOR array elements. And
"keyasint" makes a field encode to an element of CBOR map with specified int key.
@@ -23,11 +23,19 @@ The Quick Start guide is at https://github.com/fxamacker/cbor#quick-start
Function signatures identical to encoding/json include:
Marshal, Unmarshal, NewEncoder, NewDecoder, (*Encoder).Encode, (*Decoder).Decode.
Marshal, Unmarshal, NewEncoder, NewDecoder, (*Encoder).Encode, (*Decoder).Decode
Standard interfaces include:
BinaryMarshaler, BinaryUnmarshaler, Marshaler, and Unmarshaler.
BinaryMarshaler, BinaryUnmarshaler, Marshaler, and Unmarshaler
Diagnostic functions translate CBOR data item into Diagnostic Notation:
Diagnose, DiagnoseFirst
Functions that simplify using CBOR Sequences (RFC 8742) include:
UnmarshalFirst
Custom encoding and decoding is possible by implementing standard interfaces for
user-defined Go types.
@@ -50,19 +58,19 @@ Modes are intended to be reused and are safe for concurrent use.
EncMode and DecMode Interfaces
// EncMode interface uses immutable options and is safe for concurrent use.
type EncMode interface {
// EncMode interface uses immutable options and is safe for concurrent use.
type EncMode interface {
Marshal(v interface{}) ([]byte, error)
NewEncoder(w io.Writer) *Encoder
EncOptions() EncOptions // returns copy of options
}
}
// DecMode interface uses immutable options and is safe for concurrent use.
type DecMode interface {
// DecMode interface uses immutable options and is safe for concurrent use.
type DecMode interface {
Unmarshal(data []byte, v interface{}) error
NewDecoder(r io.Reader) *Decoder
DecOptions() DecOptions // returns copy of options
}
}
Using Default Encoding Mode
@@ -78,6 +86,16 @@ Using Default Decoding Mode
decoder := cbor.NewDecoder(r)
err = decoder.Decode(&v)
Using Default Mode of UnmarshalFirst to Decode CBOR Sequences
// Decode the first CBOR data item and return remaining bytes:
rest, err = cbor.UnmarshalFirst(b, &v) // decode []byte b to v
Using Extended Diagnostic Notation (EDN) to represent CBOR data
// Translate the first CBOR data item into text and return remaining bytes.
text, rest, err = cbor.DiagnoseFirst(b) // decode []byte b to text
Creating and Using Encoding Modes
// Create EncOptions using either struct literal or a function.
@@ -111,15 +129,20 @@ Decoding Options: https://github.com/fxamacker/cbor#decoding-options
Struct tags like `cbor:"name,omitempty"` and `json:"name,omitempty"` work as expected.
If both struct tags are specified then `cbor` is used.
Struct tags like "keyasint", "toarray", and "omitempty" make it easy to use
Struct tag options like "keyasint", "toarray", "omitempty", and "omitzero" make it easy to use
very compact formats like COSE and CWT (CBOR Web Tokens) with structs.
The "omitzero" option omits zero values from encoding, matching
[stdlib encoding/json behavior](https://pkg.go.dev/encoding/json#Marshal).
When specified in the `cbor` tag, the option is always honored.
When specified in the `json` tag, the option is honored when building with Go 1.24+.
For example, "toarray" makes struct fields encode to array elements. And "keyasint"
makes struct fields encode to elements of CBOR map with int keys.
https://raw.githubusercontent.com/fxamacker/images/master/cbor/v2.0.0/cbor_easy_api.png
Struct tags are listed at https://github.com/fxamacker/cbor#struct-tags-1
Struct tag options are listed at https://github.com/fxamacker/cbor#struct-tags-1
# Tests and Fuzzing

View File

@@ -58,8 +58,10 @@ import (
//
// Marshal supports format string stored under the "cbor" key in the struct
// field's tag. CBOR format string can specify the name of the field,
// "omitempty" and "keyasint" options, and special case "-" for field omission.
// If "cbor" key is absent, Marshal uses "json" key.
// "omitempty", "omitzero" and "keyasint" options, and special case "-" for
// field omission. If "cbor" key is absent, Marshal uses "json" key.
// When using the "json" key, the "omitzero" option is honored when building
// with Go 1.24+ to match stdlib encoding/json behavior.
//
// Struct field name is treated as integer if it has "keyasint" option in
// its format string. The format string must specify an integer as its
@@ -67,8 +69,8 @@ import (
//
// Special struct field "_" is used to specify struct level options, such as
// "toarray". "toarray" option enables Go struct to be encoded as CBOR array.
// "omitempty" is disabled by "toarray" to ensure that the same number
// of elements are encoded every time.
// "omitempty" and "omitzero" are disabled by "toarray" to ensure that the
// same number of elements are encoded every time.
//
// Anonymous struct fields are marshaled as if their exported fields
// were fields in the outer struct. Marshal follows the same struct fields
@@ -92,7 +94,7 @@ import (
//
// Values of other types cannot be encoded in CBOR. Attempting
// to encode such a value causes Marshal to return an UnsupportedTypeError.
func Marshal(v interface{}) ([]byte, error) {
func Marshal(v any) ([]byte, error) {
return defaultEncMode.Marshal(v)
}
@@ -103,7 +105,7 @@ func Marshal(v interface{}) ([]byte, error) {
// partially encoded data if error is returned.
//
// See Marshal for more details.
func MarshalToBuffer(v interface{}, buf *bytes.Buffer) error {
func MarshalToBuffer(v any, buf *bytes.Buffer) error {
return defaultEncMode.MarshalToBuffer(v, buf)
}
@@ -130,6 +132,20 @@ func (e *MarshalerError) Unwrap() error {
return e.err
}
type TranscodeError struct {
err error
rtype reflect.Type
sourceFormat, targetFormat string
}
func (e TranscodeError) Error() string {
return "cbor: cannot transcode from " + e.sourceFormat + " to " + e.targetFormat + ": " + e.err.Error()
}
func (e TranscodeError) Unwrap() error {
return e.err
}
// UnsupportedTypeError is returned by Marshal when attempting to encode value
// of an unsupported type.
type UnsupportedTypeError struct {
@@ -291,24 +307,51 @@ func (icm InfConvertMode) valid() bool {
return icm >= 0 && icm < maxInfConvert
}
// TimeMode specifies how to encode time.Time values.
// TimeMode specifies how to encode time.Time values in compliance with RFC 8949 (CBOR):
// - Section 3.4.1: Standard Date/Time String
// - Section 3.4.2: Epoch-Based Date/Time
// For more info, see:
// - https://www.rfc-editor.org/rfc/rfc8949.html
// NOTE: User applications that prefer to encode time with fractional seconds to an integer
// (instead of floating point or text string) can use a CBOR tag number not assigned by IANA:
// 1. Define a user-defined type in Go with just a time.Time or int64 as its data.
// 2. Implement the cbor.Marshaler and cbor.Unmarshaler interface for that user-defined type
// to encode or decode the tagged data item with an enclosed integer content.
type TimeMode int
const (
// TimeUnix causes time.Time to be encoded as epoch time in integer with second precision.
// TimeUnix causes time.Time to encode to a CBOR time (tag 1) with an integer content
// representing seconds elapsed (with 1-second precision) since UNIX Epoch UTC.
// The TimeUnix option is location independent and has a clear precision guarantee.
TimeUnix TimeMode = iota
// TimeUnixMicro causes time.Time to be encoded as epoch time in float-point rounded to microsecond precision.
// TimeUnixMicro causes time.Time to encode to a CBOR time (tag 1) with a floating point content
// representing seconds elapsed (with up to 1-microsecond precision) since UNIX Epoch UTC.
// NOTE: The floating point content is encoded to the shortest floating-point encoding that preserves
// the 64-bit floating point value. I.e., the floating point encoding can be IEEE 764:
// binary64, binary32, or binary16 depending on the content's value.
TimeUnixMicro
// TimeUnixDynamic causes time.Time to be encoded as integer if time.Time doesn't have fractional seconds,
// otherwise float-point rounded to microsecond precision.
// TimeUnixDynamic causes time.Time to encode to a CBOR time (tag 1) with either an integer content or
// a floating point content, depending on the content's value. This option is equivalent to dynamically
// choosing TimeUnix if time.Time doesn't have fractional seconds, and using TimeUnixMicro if time.Time
// has fractional seconds.
TimeUnixDynamic
// TimeRFC3339 causes time.Time to be encoded as RFC3339 formatted string with second precision.
// TimeRFC3339 causes time.Time to encode to a CBOR time (tag 0) with a text string content
// representing the time using 1-second precision in RFC3339 format. If the time.Time has a
// non-UTC timezone then a "localtime - UTC" numeric offset will be included as specified in RFC3339.
// NOTE: User applications can avoid including the RFC3339 numeric offset by:
// - providing a time.Time value set to UTC, or
// - using the TimeUnix, TimeUnixMicro, or TimeUnixDynamic option instead of TimeRFC3339.
TimeRFC3339
// TimeRFC3339Nano causes time.Time to be encoded as RFC3339 formatted string with nanosecond precision.
// TimeRFC3339Nano causes time.Time to encode to a CBOR time (tag 0) with a text string content
// representing the time using 1-nanosecond precision in RFC3339 format. If the time.Time has a
// non-UTC timezone then a "localtime - UTC" numeric offset will be included as specified in RFC3339.
// NOTE: User applications can avoid including the RFC3339 numeric offset by:
// - providing a time.Time value set to UTC, or
// - using the TimeUnix, TimeUnixMicro, or TimeUnixDynamic option instead of TimeRFC3339Nano.
TimeRFC3339Nano
maxTimeMode
@@ -481,6 +524,24 @@ func (bmm BinaryMarshalerMode) valid() bool {
return bmm >= 0 && bmm < maxBinaryMarshalerMode
}
// TextMarshalerMode specifies how to encode types that implement encoding.TextMarshaler.
type TextMarshalerMode int
const (
// TextMarshalerNone does not recognize TextMarshaler implementations during encode.
// This is the default behavior.
TextMarshalerNone TextMarshalerMode = iota
// TextMarshalerTextString encodes the output of MarshalText to a CBOR text string.
TextMarshalerTextString
maxTextMarshalerMode
)
func (tmm TextMarshalerMode) valid() bool {
return tmm >= 0 && tmm < maxTextMarshalerMode
}
// EncOptions specifies encoding options.
type EncOptions struct {
// Sort specifies sorting order.
@@ -538,6 +599,14 @@ type EncOptions struct {
// BinaryMarshaler specifies how to encode types that implement encoding.BinaryMarshaler.
BinaryMarshaler BinaryMarshalerMode
// TextMarshaler specifies how to encode types that implement encoding.TextMarshaler.
TextMarshaler TextMarshalerMode
// JSONMarshalerTranscoder sets the transcoding scheme used to marshal types that implement
// json.Marshaler but do not also implement cbor.Marshaler. If nil, encoding behavior is not
// influenced by whether or not a type implements json.Marshaler.
JSONMarshalerTranscoder Transcoder
}
// CanonicalEncOptions returns EncOptions for "Canonical CBOR" encoding,
@@ -748,6 +817,9 @@ func (opts EncOptions) encMode() (*encMode, error) { //nolint:gocritic // ignore
if !opts.BinaryMarshaler.valid() {
return nil, errors.New("cbor: invalid BinaryMarshaler " + strconv.Itoa(int(opts.BinaryMarshaler)))
}
if !opts.TextMarshaler.valid() {
return nil, errors.New("cbor: invalid TextMarshaler " + strconv.Itoa(int(opts.TextMarshaler)))
}
em := encMode{
sort: opts.Sort,
shortestFloat: opts.ShortestFloat,
@@ -767,13 +839,15 @@ func (opts EncOptions) encMode() (*encMode, error) { //nolint:gocritic // ignore
byteSliceLaterEncodingTag: byteSliceLaterEncodingTag,
byteArray: opts.ByteArray,
binaryMarshaler: opts.BinaryMarshaler,
textMarshaler: opts.TextMarshaler,
jsonMarshalerTranscoder: opts.JSONMarshalerTranscoder,
}
return &em, nil
}
// EncMode is the main interface for CBOR encoding.
type EncMode interface {
Marshal(v interface{}) ([]byte, error)
Marshal(v any) ([]byte, error)
NewEncoder(w io.Writer) *Encoder
EncOptions() EncOptions
}
@@ -783,7 +857,7 @@ type EncMode interface {
// into the built-in buffer pool.
type UserBufferEncMode interface {
EncMode
MarshalToBuffer(v interface{}, buf *bytes.Buffer) error
MarshalToBuffer(v any, buf *bytes.Buffer) error
// This private method is to prevent users implementing
// this interface and so future additions to it will
@@ -812,6 +886,8 @@ type encMode struct {
byteSliceLaterEncodingTag uint64
byteArray ByteArrayMode
binaryMarshaler BinaryMarshalerMode
textMarshaler TextMarshalerMode
jsonMarshalerTranscoder Transcoder
}
var defaultEncMode, _ = EncOptions{}.encMode()
@@ -888,22 +964,24 @@ func getMarshalerDecMode(indefLength IndefLengthMode, tagsMd TagsMode) *decMode
// EncOptions returns user specified options used to create this EncMode.
func (em *encMode) EncOptions() EncOptions {
return EncOptions{
Sort: em.sort,
ShortestFloat: em.shortestFloat,
NaNConvert: em.nanConvert,
InfConvert: em.infConvert,
BigIntConvert: em.bigIntConvert,
Time: em.time,
TimeTag: em.timeTag,
IndefLength: em.indefLength,
NilContainers: em.nilContainers,
TagsMd: em.tagsMd,
OmitEmpty: em.omitEmpty,
String: em.stringType,
FieldName: em.fieldName,
ByteSliceLaterFormat: em.byteSliceLaterFormat,
ByteArray: em.byteArray,
BinaryMarshaler: em.binaryMarshaler,
Sort: em.sort,
ShortestFloat: em.shortestFloat,
NaNConvert: em.nanConvert,
InfConvert: em.infConvert,
BigIntConvert: em.bigIntConvert,
Time: em.time,
TimeTag: em.timeTag,
IndefLength: em.indefLength,
NilContainers: em.nilContainers,
TagsMd: em.tagsMd,
OmitEmpty: em.omitEmpty,
String: em.stringType,
FieldName: em.fieldName,
ByteSliceLaterFormat: em.byteSliceLaterFormat,
ByteArray: em.byteArray,
BinaryMarshaler: em.binaryMarshaler,
TextMarshaler: em.textMarshaler,
JSONMarshalerTranscoder: em.jsonMarshalerTranscoder,
}
}
@@ -921,7 +999,7 @@ func (em *encMode) encTagBytes(t reflect.Type) []byte {
// Marshal returns the CBOR encoding of v using em encoding mode.
//
// See the documentation for Marshal for details.
func (em *encMode) Marshal(v interface{}) ([]byte, error) {
func (em *encMode) Marshal(v any) ([]byte, error) {
e := getEncodeBuffer()
if err := encode(e, em, reflect.ValueOf(v)); err != nil {
@@ -943,7 +1021,7 @@ func (em *encMode) Marshal(v interface{}) ([]byte, error) {
// partially encoded data if error is returned.
//
// See Marshal for more details.
func (em *encMode) MarshalToBuffer(v interface{}, buf *bytes.Buffer) error {
func (em *encMode) MarshalToBuffer(v any, buf *bytes.Buffer) error {
if buf == nil {
return fmt.Errorf("cbor: encoding buffer provided by user is nil")
}
@@ -957,7 +1035,7 @@ func (em *encMode) NewEncoder(w io.Writer) *Encoder {
// encodeBufferPool caches unused bytes.Buffer objects for later reuse.
var encodeBufferPool = sync.Pool{
New: func() interface{} {
New: func() any {
e := new(bytes.Buffer)
e.Grow(32) // TODO: make this configurable
return e
@@ -975,6 +1053,7 @@ func putEncodeBuffer(e *bytes.Buffer) {
type encodeFunc func(e *bytes.Buffer, em *encMode, v reflect.Value) error
type isEmptyFunc func(em *encMode, v reflect.Value) (empty bool, err error)
type isZeroFunc func(v reflect.Value) (zero bool, err error)
func encode(e *bytes.Buffer, em *encMode, v reflect.Value) error {
if !v.IsValid() {
@@ -983,7 +1062,7 @@ func encode(e *bytes.Buffer, em *encMode, v reflect.Value) error {
return nil
}
vt := v.Type()
f, _ := getEncodeFunc(vt)
f, _, _ := getEncodeFunc(vt)
if f == nil {
return &UnsupportedTypeError{vt}
}
@@ -1483,6 +1562,15 @@ func encodeStruct(e *bytes.Buffer, em *encMode, v reflect.Value) (err error) {
continue
}
}
if f.omitZero {
zero, err := f.izf(fv)
if err != nil {
return err
}
if zero {
continue
}
}
if !f.keyAsInt && em.fieldName == FieldNameToByteString {
e.Write(f.cborNameByteString)
@@ -1665,6 +1753,107 @@ func (bme binaryMarshalerEncoder) isEmpty(em *encMode, v reflect.Value) (bool, e
return len(data) == 0, nil
}
type textMarshalerEncoder struct {
alternateEncode encodeFunc
alternateIsEmpty isEmptyFunc
}
func (tme textMarshalerEncoder) encode(e *bytes.Buffer, em *encMode, v reflect.Value) error {
if em.textMarshaler == TextMarshalerNone {
return tme.alternateEncode(e, em, v)
}
vt := v.Type()
m, ok := v.Interface().(encoding.TextMarshaler)
if !ok {
pv := reflect.New(vt)
pv.Elem().Set(v)
m = pv.Interface().(encoding.TextMarshaler)
}
data, err := m.MarshalText()
if err != nil {
return fmt.Errorf("cbor: cannot marshal text for %s: %w", vt, err)
}
if b := em.encTagBytes(vt); b != nil {
e.Write(b)
}
encodeHead(e, byte(cborTypeTextString), uint64(len(data)))
e.Write(data)
return nil
}
func (tme textMarshalerEncoder) isEmpty(em *encMode, v reflect.Value) (bool, error) {
if em.textMarshaler == TextMarshalerNone {
return tme.alternateIsEmpty(em, v)
}
m, ok := v.Interface().(encoding.TextMarshaler)
if !ok {
pv := reflect.New(v.Type())
pv.Elem().Set(v)
m = pv.Interface().(encoding.TextMarshaler)
}
data, err := m.MarshalText()
if err != nil {
return false, fmt.Errorf("cbor: cannot marshal text for %s: %w", v.Type(), err)
}
return len(data) == 0, nil
}
type jsonMarshalerEncoder struct {
alternateEncode encodeFunc
alternateIsEmpty isEmptyFunc
}
func (jme jsonMarshalerEncoder) encode(e *bytes.Buffer, em *encMode, v reflect.Value) error {
if em.jsonMarshalerTranscoder == nil {
return jme.alternateEncode(e, em, v)
}
vt := v.Type()
m, ok := v.Interface().(jsonMarshaler)
if !ok {
pv := reflect.New(vt)
pv.Elem().Set(v)
m = pv.Interface().(jsonMarshaler)
}
json, err := m.MarshalJSON()
if err != nil {
return err
}
offset := e.Len()
if b := em.encTagBytes(vt); b != nil {
e.Write(b)
}
if err := em.jsonMarshalerTranscoder.Transcode(e, bytes.NewReader(json)); err != nil {
return &TranscodeError{err: err, rtype: vt, sourceFormat: "json", targetFormat: "cbor"}
}
// Validate that the transcode function has written exactly one well-formed data item.
d := decoder{data: e.Bytes()[offset:], dm: getMarshalerDecMode(em.indefLength, em.tagsMd)}
if err := d.wellformed(false, true); err != nil {
e.Truncate(offset)
return &TranscodeError{err: err, rtype: vt, sourceFormat: "json", targetFormat: "cbor"}
}
return nil
}
func (jme jsonMarshalerEncoder) isEmpty(em *encMode, v reflect.Value) (bool, error) {
if em.jsonMarshalerTranscoder == nil {
return jme.alternateIsEmpty(em, v)
}
// As with types implementing cbor.Marshaler, transcoded json.Marshaler values always encode
// as exactly one complete CBOR data item.
return false, nil
}
func encodeMarshalerType(e *bytes.Buffer, em *encMode, v reflect.Value) error {
if em.tagsMd == TagsForbidden && v.Type() == typeRawTag {
return errors.New("cbor: cannot encode cbor.RawTag when TagsMd is TagsForbidden")
@@ -1768,41 +1957,45 @@ func encodeHead(e *bytes.Buffer, t byte, n uint64) int {
return headSize
}
type jsonMarshaler interface{ MarshalJSON() ([]byte, error) }
var (
typeMarshaler = reflect.TypeOf((*Marshaler)(nil)).Elem()
typeBinaryMarshaler = reflect.TypeOf((*encoding.BinaryMarshaler)(nil)).Elem()
typeTextMarshaler = reflect.TypeOf((*encoding.TextMarshaler)(nil)).Elem()
typeJSONMarshaler = reflect.TypeOf((*jsonMarshaler)(nil)).Elem()
typeRawMessage = reflect.TypeOf(RawMessage(nil))
typeByteString = reflect.TypeOf(ByteString(""))
)
func getEncodeFuncInternal(t reflect.Type) (ef encodeFunc, ief isEmptyFunc) {
func getEncodeFuncInternal(t reflect.Type) (ef encodeFunc, ief isEmptyFunc, izf isZeroFunc) {
k := t.Kind()
if k == reflect.Ptr {
return getEncodeIndirectValueFunc(t), isEmptyPtr
if k == reflect.Pointer {
return getEncodeIndirectValueFunc(t), isEmptyPtr, getIsZeroFunc(t)
}
switch t {
case typeSimpleValue:
return encodeMarshalerType, isEmptyUint
return encodeMarshalerType, isEmptyUint, getIsZeroFunc(t)
case typeTag:
return encodeTag, alwaysNotEmpty
return encodeTag, alwaysNotEmpty, getIsZeroFunc(t)
case typeTime:
return encodeTime, alwaysNotEmpty
return encodeTime, alwaysNotEmpty, getIsZeroFunc(t)
case typeBigInt:
return encodeBigInt, alwaysNotEmpty
return encodeBigInt, alwaysNotEmpty, getIsZeroFunc(t)
case typeRawMessage:
return encodeMarshalerType, isEmptySlice
return encodeMarshalerType, isEmptySlice, getIsZeroFunc(t)
case typeByteString:
return encodeMarshalerType, isEmptyString
return encodeMarshalerType, isEmptyString, getIsZeroFunc(t)
}
if reflect.PtrTo(t).Implements(typeMarshaler) {
return encodeMarshalerType, alwaysNotEmpty
if reflect.PointerTo(t).Implements(typeMarshaler) {
return encodeMarshalerType, alwaysNotEmpty, getIsZeroFunc(t)
}
if reflect.PtrTo(t).Implements(typeBinaryMarshaler) {
if reflect.PointerTo(t).Implements(typeBinaryMarshaler) {
defer func() {
// capture encoding method used for modes that disable BinaryMarshaler
bme := binaryMarshalerEncoder{
@@ -1813,41 +2006,65 @@ func getEncodeFuncInternal(t reflect.Type) (ef encodeFunc, ief isEmptyFunc) {
ief = bme.isEmpty
}()
}
if reflect.PointerTo(t).Implements(typeTextMarshaler) {
defer func() {
// capture encoding method used for modes that disable TextMarshaler
tme := textMarshalerEncoder{
alternateEncode: ef,
alternateIsEmpty: ief,
}
ef = tme.encode
ief = tme.isEmpty
}()
}
if reflect.PointerTo(t).Implements(typeJSONMarshaler) {
defer func() {
// capture encoding method used for modes that don't support transcoding
// from types that implement json.Marshaler.
jme := jsonMarshalerEncoder{
alternateEncode: ef,
alternateIsEmpty: ief,
}
ef = jme.encode
ief = jme.isEmpty
}()
}
switch k {
case reflect.Bool:
return encodeBool, isEmptyBool
return encodeBool, isEmptyBool, getIsZeroFunc(t)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return encodeInt, isEmptyInt
return encodeInt, isEmptyInt, getIsZeroFunc(t)
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
return encodeUint, isEmptyUint
return encodeUint, isEmptyUint, getIsZeroFunc(t)
case reflect.Float32, reflect.Float64:
return encodeFloat, isEmptyFloat
return encodeFloat, isEmptyFloat, getIsZeroFunc(t)
case reflect.String:
return encodeString, isEmptyString
return encodeString, isEmptyString, getIsZeroFunc(t)
case reflect.Slice:
if t.Elem().Kind() == reflect.Uint8 {
return encodeByteString, isEmptySlice
return encodeByteString, isEmptySlice, getIsZeroFunc(t)
}
fallthrough
case reflect.Array:
f, _ := getEncodeFunc(t.Elem())
f, _, _ := getEncodeFunc(t.Elem())
if f == nil {
return nil, nil
return nil, nil, nil
}
return arrayEncodeFunc{f: f}.encode, isEmptySlice
return arrayEncodeFunc{f: f}.encode, isEmptySlice, getIsZeroFunc(t)
case reflect.Map:
f := getEncodeMapFunc(t)
if f == nil {
return nil, nil
return nil, nil, nil
}
return f, isEmptyMap
return f, isEmptyMap, getIsZeroFunc(t)
case reflect.Struct:
// Get struct's special field "_" tag options
@@ -1855,31 +2072,31 @@ func getEncodeFuncInternal(t reflect.Type) (ef encodeFunc, ief isEmptyFunc) {
tag := f.Tag.Get("cbor")
if tag != "-" {
if hasToArrayOption(tag) {
return encodeStructToArray, isEmptyStruct
return encodeStructToArray, isEmptyStruct, isZeroFieldStruct
}
}
}
return encodeStruct, isEmptyStruct
return encodeStruct, isEmptyStruct, getIsZeroFunc(t)
case reflect.Interface:
return encodeIntf, isEmptyIntf
return encodeIntf, isEmptyIntf, getIsZeroFunc(t)
}
return nil, nil
return nil, nil, nil
}
func getEncodeIndirectValueFunc(t reflect.Type) encodeFunc {
for t.Kind() == reflect.Ptr {
for t.Kind() == reflect.Pointer {
t = t.Elem()
}
f, _ := getEncodeFunc(t)
f, _, _ := getEncodeFunc(t)
if f == nil {
return nil
}
return func(e *bytes.Buffer, em *encMode, v reflect.Value) error {
for v.Kind() == reflect.Ptr && !v.IsNil() {
for v.Kind() == reflect.Pointer && !v.IsNil() {
v = v.Elem()
}
if v.Kind() == reflect.Ptr && v.IsNil() {
if v.Kind() == reflect.Pointer && v.IsNil() {
e.Write(cborNil)
return nil
}
@@ -1987,3 +2204,96 @@ func float32NaNFromReflectValue(v reflect.Value) float32 {
f32 := p.Convert(reflect.TypeOf((*float32)(nil))).Elem().Interface().(float32)
return f32
}
type isZeroer interface {
IsZero() bool
}
var isZeroerType = reflect.TypeOf((*isZeroer)(nil)).Elem()
// getIsZeroFunc returns a function for the given type that can be called to determine if a given value is zero.
// Types that implement `IsZero() bool` are delegated to for non-nil values.
// Types that do not implement `IsZero() bool` use the reflect.Value#IsZero() implementation.
// The returned function matches behavior of stdlib encoding/json behavior in Go 1.24+.
func getIsZeroFunc(t reflect.Type) isZeroFunc {
// Provide a function that uses a type's IsZero method if defined.
switch {
case t == nil:
return isZeroDefault
case t.Kind() == reflect.Interface && t.Implements(isZeroerType):
return isZeroInterfaceCustom
case t.Kind() == reflect.Pointer && t.Implements(isZeroerType):
return isZeroPointerCustom
case t.Implements(isZeroerType):
return isZeroCustom
case reflect.PointerTo(t).Implements(isZeroerType):
return isZeroAddrCustom
default:
return isZeroDefault
}
}
// isZeroInterfaceCustom returns true for nil or pointer-to-nil values,
// and delegates to the custom IsZero() implementation otherwise.
func isZeroInterfaceCustom(v reflect.Value) (bool, error) {
kind := v.Kind()
switch kind {
case reflect.Chan, reflect.Func, reflect.Map, reflect.Pointer, reflect.Interface, reflect.Slice:
if v.IsNil() {
return true, nil
}
}
switch kind {
case reflect.Interface, reflect.Pointer:
if elem := v.Elem(); elem.Kind() == reflect.Pointer && elem.IsNil() {
return true, nil
}
}
return v.Interface().(isZeroer).IsZero(), nil
}
// isZeroPointerCustom returns true for nil values,
// and delegates to the custom IsZero() implementation otherwise.
func isZeroPointerCustom(v reflect.Value) (bool, error) {
if v.IsNil() {
return true, nil
}
return v.Interface().(isZeroer).IsZero(), nil
}
// isZeroCustom delegates to the custom IsZero() implementation.
func isZeroCustom(v reflect.Value) (bool, error) {
return v.Interface().(isZeroer).IsZero(), nil
}
// isZeroAddrCustom delegates to the custom IsZero() implementation of the addr of the value.
func isZeroAddrCustom(v reflect.Value) (bool, error) {
if !v.CanAddr() {
// Temporarily box v so we can take the address.
v2 := reflect.New(v.Type()).Elem()
v2.Set(v)
v = v2
}
return v.Addr().Interface().(isZeroer).IsZero(), nil
}
// isZeroDefault calls reflect.Value#IsZero()
func isZeroDefault(v reflect.Value) (bool, error) {
if !v.IsValid() {
// v is zero value
return true, nil
}
return v.IsZero(), nil
}
// isZeroFieldStruct is used to determine whether to omit toarray structs
func isZeroFieldStruct(v reflect.Value) (bool, error) {
structType, err := getEncodingStructType(v.Type())
if err != nil {
return false, err
}
return len(structType.fields) == 0, nil
}

View File

@@ -1,8 +1,6 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
//go:build go1.20
package cbor
import (
@@ -67,8 +65,8 @@ func (me *mapKeyValueEncodeFunc) encodeKeyValues(e *bytes.Buffer, em *encMode, v
}
func getEncodeMapFunc(t reflect.Type) encodeFunc {
kf, _ := getEncodeFunc(t.Key())
ef, _ := getEncodeFunc(t.Elem())
kf, _, _ := getEncodeFunc(t.Key())
ef, _, _ := getEncodeFunc(t.Elem())
if kf == nil || ef == nil {
return nil
}
@@ -76,13 +74,13 @@ func getEncodeMapFunc(t reflect.Type) encodeFunc {
kf: kf,
ef: ef,
kpool: sync.Pool{
New: func() interface{} {
New: func() any {
rk := reflect.New(t.Key()).Elem()
return &rk
},
},
vpool: sync.Pool{
New: func() interface{} {
New: func() any {
rv := reflect.New(t.Elem()).Elem()
return &rv
},

View File

@@ -1,60 +0,0 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
//go:build !go1.20
package cbor
import (
"bytes"
"reflect"
)
type mapKeyValueEncodeFunc struct {
kf, ef encodeFunc
}
func (me *mapKeyValueEncodeFunc) encodeKeyValues(e *bytes.Buffer, em *encMode, v reflect.Value, kvs []keyValue) error {
if kvs == nil {
for i, iter := 0, v.MapRange(); iter.Next(); i++ {
if err := me.kf(e, em, iter.Key()); err != nil {
return err
}
if err := me.ef(e, em, iter.Value()); err != nil {
return err
}
}
return nil
}
initial := e.Len()
for i, iter := 0, v.MapRange(); iter.Next(); i++ {
offset := e.Len()
if err := me.kf(e, em, iter.Key()); err != nil {
return err
}
valueOffset := e.Len()
if err := me.ef(e, em, iter.Value()); err != nil {
return err
}
kvs[i] = keyValue{
offset: offset - initial,
valueOffset: valueOffset - initial,
nextOffset: e.Len() - initial,
}
}
return nil
}
func getEncodeMapFunc(t reflect.Type) encodeFunc {
kf, _ := getEncodeFunc(t.Key())
ef, _ := getEncodeFunc(t.Elem())
if kf == nil || ef == nil {
return nil
}
mkv := &mapKeyValueEncodeFunc{kf: kf, ef: ef}
return mapEncodeFunc{
e: mkv.encodeKeyValues,
}.encode
}

View File

@@ -0,0 +1,8 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
//go:build go1.24
package cbor
var jsonStdlibSupportsOmitzero = true

View File

@@ -0,0 +1,8 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
//go:build !go1.24
package cbor
var jsonStdlibSupportsOmitzero = false

View File

@@ -1,3 +1,6 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
package cbor
import (
@@ -45,6 +48,9 @@ func (sv SimpleValue) MarshalCBOR() ([]byte, error) {
}
// UnmarshalCBOR decodes CBOR simple value (major type 7) to SimpleValue.
//
// Deprecated: No longer used by this codec; kept for compatibility
// with user apps that directly call this function.
func (sv *SimpleValue) UnmarshalCBOR(data []byte) error {
if sv == nil {
return errors.New("cbor.SimpleValue: UnmarshalCBOR on nil pointer")
@@ -52,6 +58,29 @@ func (sv *SimpleValue) UnmarshalCBOR(data []byte) error {
d := decoder{data: data, dm: defaultDecMode}
// Check well-formedness of CBOR data item.
// SimpleValue.UnmarshalCBOR() is exported, so
// the codec needs to support same behavior for:
// - Unmarshal(data, *SimpleValue)
// - SimpleValue.UnmarshalCBOR(data)
err := d.wellformed(false, false)
if err != nil {
return err
}
return sv.unmarshalCBOR(data)
}
// unmarshalCBOR decodes CBOR simple value (major type 7) to SimpleValue.
// This function assumes data is well-formed, and does not perform bounds checking.
// This function is called by Unmarshal().
func (sv *SimpleValue) unmarshalCBOR(data []byte) error {
if sv == nil {
return errors.New("cbor.SimpleValue: UnmarshalCBOR on nil pointer")
}
d := decoder{data: data, dm: defaultDecMode}
typ, ai, val := d.getHead()
if typ != cborTypePrimitives {

View File

@@ -26,7 +26,7 @@ func NewDecoder(r io.Reader) *Decoder {
}
// Decode reads CBOR value and decodes it into the value pointed to by v.
func (dec *Decoder) Decode(v interface{}) error {
func (dec *Decoder) Decode(v any) error {
_, err := dec.readNext()
if err != nil {
// Return validation error or read error.
@@ -170,7 +170,7 @@ func NewEncoder(w io.Writer) *Encoder {
}
// Encode writes the CBOR encoding of v.
func (enc *Encoder) Encode(v interface{}) error {
func (enc *Encoder) Encode(v any) error {
if len(enc.indefTypes) > 0 && v != nil {
indefType := enc.indefTypes[len(enc.indefTypes)-1]
if indefType == cborTypeTextString {

View File

@@ -18,9 +18,11 @@ type field struct {
typ reflect.Type
ef encodeFunc
ief isEmptyFunc
izf isZeroFunc
typInfo *typeInfo // used to decoder to reuse type info
tagged bool // used to choose dominant field (at the same level tagged fields dominate untagged fields)
omitEmpty bool // used to skip empty field
omitZero bool // used to skip zero field
keyAsInt bool // used to encode/decode field name as int
}
@@ -157,7 +159,7 @@ func appendFields(
f := t.Field(i)
ft := f.Type
for ft.Kind() == reflect.Ptr {
for ft.Kind() == reflect.Pointer {
ft = ft.Elem()
}
@@ -165,9 +167,11 @@ func appendFields(
continue
}
cborTag := true
tag := f.Tag.Get("cbor")
if tag == "" {
tag = f.Tag.Get("json")
cborTag = false
}
if tag == "-" {
continue
@@ -177,7 +181,7 @@ func appendFields(
// Parse field tag options
var tagFieldName string
var omitempty, keyasint bool
var omitempty, omitzero, keyasint bool
for j := 0; tag != ""; j++ {
var token string
idx := strings.IndexByte(tag, ',')
@@ -192,6 +196,10 @@ func appendFields(
switch token {
case "omitempty":
omitempty = true
case "omitzero":
if cborTag || jsonStdlibSupportsOmitzero {
omitzero = true
}
case "keyasint":
keyasint = true
}
@@ -213,6 +221,7 @@ func appendFields(
idx: fIdx,
typ: f.Type,
omitEmpty: omitempty,
omitZero: omitzero,
keyAsInt: keyasint,
tagged: tagged})
} else {
@@ -230,8 +239,7 @@ func appendFields(
// a nonexportable anonymous field of struct type.
// Nonexportable anonymous field of struct type can contain exportable fields.
func isFieldExportable(f reflect.StructField, fk reflect.Kind) bool { //nolint:gocritic // ignore hugeParam
exportable := f.PkgPath == ""
return exportable || (f.Anonymous && fk == reflect.Struct)
return f.IsExported() || (f.Anonymous && fk == reflect.Struct)
}
type embeddedFieldNullPtrFunc func(reflect.Value) (reflect.Value, error)
@@ -244,7 +252,7 @@ func getFieldValue(v reflect.Value, idx []int, f embeddedFieldNullPtrFunc) (fv r
fv = fv.Field(n)
if i < len(idx)-1 {
if fv.Kind() == reflect.Ptr && fv.Type().Elem().Kind() == reflect.Struct {
if fv.Kind() == reflect.Pointer && fv.Type().Elem().Kind() == reflect.Struct {
if fv.IsNil() {
// Null pointer to embedded struct field
fv, err = f(fv)

View File

@@ -1,3 +1,6 @@
// Copyright (c) Faye Amacker. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.
package cbor
import (
@@ -7,27 +10,54 @@ import (
"sync"
)
// Tag represents CBOR tag data, including tag number and unmarshaled tag content. Marshaling and
// unmarshaling of tag content is subject to any encode and decode options that would apply to
// enclosed data item if it were to appear outside of a tag.
// Tag represents a tagged data item (CBOR major type 6), comprising a tag number and the unmarshaled tag content.
// NOTE: The same encoding and decoding options that apply to untagged CBOR data items also applies to tag content
// during encoding and decoding.
type Tag struct {
Number uint64
Content interface{}
Content any
}
// RawTag represents CBOR tag data, including tag number and raw tag content.
// RawTag implements Unmarshaler and Marshaler interfaces.
// RawTag represents a tagged data item (CBOR major type 6), comprising a tag number and the raw tag content.
// The raw tag content (enclosed data item) is a CBOR-encoded data item.
// RawTag can be used to delay decoding a CBOR data item or precompute encoding a CBOR data item.
type RawTag struct {
Number uint64
Content RawMessage
}
// UnmarshalCBOR sets *t with tag number and raw tag content copied from data.
// UnmarshalCBOR sets *t with the tag number and the raw tag content copied from data.
//
// Deprecated: No longer used by this codec; kept for compatibility
// with user apps that directly call this function.
func (t *RawTag) UnmarshalCBOR(data []byte) error {
if t == nil {
return errors.New("cbor.RawTag: UnmarshalCBOR on nil pointer")
}
d := decoder{data: data, dm: defaultDecMode}
// Check if data is a well-formed CBOR data item.
// RawTag.UnmarshalCBOR() is exported, so
// the codec needs to support same behavior for:
// - Unmarshal(data, *RawTag)
// - RawTag.UnmarshalCBOR(data)
err := d.wellformed(false, false)
if err != nil {
return err
}
return t.unmarshalCBOR(data)
}
// unmarshalCBOR sets *t with the tag number and the raw tag content copied from data.
// This function assumes data is well-formed, and does not perform bounds checking.
// This function is called by Unmarshal().
func (t *RawTag) unmarshalCBOR(data []byte) error {
if t == nil {
return errors.New("cbor.RawTag: UnmarshalCBOR on nil pointer")
}
// Decoding CBOR null and undefined to cbor.RawTag is no-op.
if len(data) == 1 && (data[0] == 0xf6 || data[0] == 0xf7) {
return nil
@@ -193,7 +223,7 @@ func (t *syncTagSet) Add(opts TagOptions, contentType reflect.Type, num uint64,
if contentType == nil {
return errors.New("cbor: cannot add nil content type to TagSet")
}
for contentType.Kind() == reflect.Ptr {
for contentType.Kind() == reflect.Pointer {
contentType = contentType.Elem()
}
tag, err := newTagItem(opts, contentType, num, nestedNum...)
@@ -216,7 +246,7 @@ func (t *syncTagSet) Add(opts TagOptions, contentType reflect.Type, num uint64,
// Remove removes given tag content type from TagSet.
func (t *syncTagSet) Remove(contentType reflect.Type) {
for contentType.Kind() == reflect.Ptr {
for contentType.Kind() == reflect.Pointer {
contentType = contentType.Elem()
}
t.Lock()