Involved Source Filesdecode.go Package jsontext implements syntactic processing of JSON
as specified in RFC 4627, RFC 7159, RFC 7493, RFC 8259, and RFC 8785.
JSON is a simple data interchange format that can represent
primitive data types such as booleans, strings, and numbers,
in addition to structured data types such as objects and arrays.
The [Encoder] and [Decoder] types are used to encode or decode
a stream of JSON tokens or values.
# Tokens and Values
A JSON token refers to the basic structural elements of JSON:
- a JSON literal (i.e., null, true, or false)
- a JSON string (e.g., "hello, world!")
- a JSON number (e.g., 123.456)
- a begin or end delimiter for a JSON object (i.e., '{' or '}')
- a begin or end delimiter for a JSON array (i.e., '[' or ']')
A JSON token is represented by the [Token] type in Go. Technically,
there are two additional structural characters (i.e., ':' and ','),
but there is no [Token] representation for them since their presence
can be inferred by the structure of the JSON grammar itself.
For example, there must always be an implicit colon between
the name and value of a JSON object member.
A JSON value refers to a complete unit of JSON data:
- a JSON literal, string, or number
- a JSON object (e.g., `{"name":"value"}`)
- a JSON array (e.g., `[1,2,3]`)
A JSON value is represented by the [Value] type in Go and is a []byte
containing the raw textual representation of the value. There is some overlap
between tokens and values as both contain literals, strings, and numbers.
However, only a value can represent the entirety of a JSON object or array.
The [Encoder] and [Decoder] types contain methods to read or write the next
[Token] or [Value] in a sequence. They maintain a state machine to validate
whether the sequence of JSON tokens and/or values produces valid JSON.
[Options] may be passed to the [NewEncoder] or [NewDecoder] constructors
to configure the behavior of encoding and decoding.
# Terminology
The terms "encode" and "decode" are used for syntactic functionality
that is concerned with processing JSON based on its grammar, and
the terms "marshal" and "unmarshal" are used for semantic functionality
that determines the meaning of JSON values as Go values and vice versa.
This package deals with JSON syntax,
while [encoding/json/v2] deals with JSON semantics.
The goal is to provide a clear distinction between functionality that
is purely concerned with encoding versus that of marshaling.
For example, one can directly encode a stream of JSON tokens without
needing to marshal a concrete Go value representing them.
Similarly, one can decode a stream of JSON tokens without
needing to unmarshal them into a concrete Go value.
This package uses JSON terminology when discussing JSON, which may differ
from related concepts in Go or elsewhere in computing literature.
- a JSON "object" refers to an unordered collection of name/value members.
- a JSON "array" refers to an ordered sequence of elements.
- a JSON "value" refers to either a literal (i.e., null, false, or true),
string, number, object, or array.
See RFC 8259 for more information.
# Specifications
Relevant specifications include RFC 4627, RFC 7159, RFC 7493, RFC 8259,
and RFC 8785. Each RFC is generally a stricter subset of another RFC.
In increasing order of strictness:
- RFC 4627 and RFC 7159 do not require (but recommend) the use of UTF-8
and also do not require (but recommend) that object names be unique.
- RFC 8259 requires the use of UTF-8,
but does not require (but recommends) that object names be unique.
- RFC 7493 requires the use of UTF-8
and also requires that object names be unique.
- RFC 8785 defines a canonical representation. It requires the use of UTF-8
and also requires that object names be unique and in a specific ordering.
It specifies exactly how strings and numbers must be formatted.
The primary difference between RFC 4627 and RFC 7159 is that the former
restricted top-level values to only JSON objects and arrays, while
RFC 7159 and subsequent RFCs permit top-level values to additionally be
JSON nulls, booleans, strings, or numbers.
By default, this package operates on RFC 7493, but can be configured
to operate according to the other RFC specifications.
RFC 7493 is a stricter subset of RFC 8259 and fully compliant with it.
In particular, it makes specific choices about behavior that RFC 8259
leaves as undefined in order to ensure greater interoperability.
# Security Considerations
See the "Security Considerations" section in [encoding/json/v2].encode.goerrors.goexport.gooptions.gopools.goquote.gostate.gotoken.govalue.go
Code Examples
package main
import (
"fmt"
"log"
"encoding/json/jsontext"
"encoding/json/v2"
)
func main() {
page := struct {
Title string
Body string
}{
Title: "Example Embedded Javascript",
Body: `<script> console.log("Hello, world!"); </script>`,
}
b, err := json.Marshal(&page,
// Escape certain runes within a JSON string so that
// JSON will be safe to directly embed inside HTML.
jsontext.EscapeForHTML(true),
jsontext.EscapeForJS(true),
jsontext.Multiline(true)) // expand for readability
if err != nil {
log.Fatal(err)
}
fmt.Println(string(b))
}
package main
import (
"fmt"
"log"
"encoding/json/jsontext"
"encoding/json/v2"
)
func main() {
type Pet struct {
Name string
Species string
Breed string
}
p := Pet{
Name: "Oliver",
Species: "Dog",
Breed: "Goldendoodle",
}
b, err := json.Marshal(p, jsontext.Multiline(true))
if err != nil {
log.Fatal(err)
}
fmt.Println(string(b))
}
package main
import (
"bytes"
"fmt"
"io"
"log"
"strings"
"encoding/json/jsontext"
)
func main() {
// Example input with non-idiomatic use of "Golang" instead of "Go".
const input = `{
"title": "Golang version 1 is released",
"author": "Andrew Gerrand",
"date": "2012-03-28",
"text": "Today marks a major milestone in the development of the Golang programming language.",
"otherArticles": [
"Twelve Years of Golang",
"The Laws of Reflection",
"Learn Golang from your browser"
]
}`
// Using a Decoder and Encoder, we can parse through every token,
// check and modify the token if necessary, and
// write the token to the output.
var replacements []jsontext.Pointer
in := strings.NewReader(input)
dec := jsontext.NewDecoder(in)
out := new(bytes.Buffer)
enc := jsontext.NewEncoder(out, jsontext.Multiline(true)) // expand for readability
for {
// Read a token from the input.
tok, err := dec.ReadToken()
if err != nil {
if err == io.EOF {
break
}
log.Fatal(err)
}
// Check whether the token contains the string "Golang" and
// replace each occurrence with "Go" instead.
if tok.Kind() == '"' && strings.Contains(tok.String(), "Golang") {
replacements = append(replacements, dec.StackPointer())
tok = jsontext.String(strings.ReplaceAll(tok.String(), "Golang", "Go"))
}
// Write the (possibly modified) token to the output.
if err := enc.WriteToken(tok); err != nil {
log.Fatal(err)
}
}
// Print the list of replacements and the adjusted JSON output.
if len(replacements) > 0 {
fmt.Println(`Replaced "Golang" with "Go" in:`)
for _, where := range replacements {
fmt.Println("\t" + where)
}
fmt.Println()
}
fmt.Println("Result:", out.String())
}
Package-Level Type Names (total 27, in which 8 are exported)
/* sort exporteds by: | */
Decoder is a streaming decoder for raw JSON tokens and values.
It is used to read a stream of top-level JSON values,
each separated by optional whitespace characters.
[Decoder.ReadToken] and [Decoder.ReadValue] calls may be interleaved.
For example, the following JSON value:
{"name":"value","array":[null,false,true,3.14159],"object":{"k":"v"}}
can be parsed with the following calls (ignoring errors for brevity):
d.ReadToken() // {
d.ReadToken() // "name"
d.ReadToken() // "value"
d.ReadValue() // "array"
d.ReadToken() // [
d.ReadToken() // null
d.ReadToken() // false
d.ReadValue() // true
d.ReadToken() // 3.14159
d.ReadToken() // ]
d.ReadValue() // "object"
d.ReadValue() // {"k":"v"}
d.ReadToken() // }
The above is one of many possible sequences of calls and
may not represent the most sensible method to call for any given token/value.
For example, it is probably more common to call [Decoder.ReadToken] to obtain a
string token for object names.sdecoderState InputOffset returns the current input byte offset. It gives the location
of the next byte immediately after the most recently returned token or value.
The number of bytes actually read from the underlying [io.Reader] may be more
than this offset due to internal buffering. Options returns the options used to construct the decoder and
may additionally contain semantic options passed to a
[encoding/json/v2.UnmarshalDecode] call.
If operating within
a [encoding/json/v2.UnmarshalerFrom.UnmarshalJSONFrom] method call or
a [encoding/json/v2.UnmarshalFromFunc] function call,
then the returned options are only valid within the call. PeekKind retrieves the next token kind, but does not advance the read offset.
It returns [KindInvalid] if an error occurs. Any such error is cached until
the next read call and it is the caller's responsibility to eventually
follow up a PeekKind call with a read call. ReadToken reads the next [Token], advancing the read offset.
The returned token is only valid until the next Peek, Read, or Skip call.
It returns [io.EOF] if there are no more tokens. ReadValue returns the next raw JSON value, advancing the read offset.
The value is stripped of any leading or trailing whitespace and
contains the exact bytes of the input, which may contain invalid UTF-8
if [AllowInvalidUTF8] is specified.
The returned value is only valid until the next Peek, Read, or Skip call and
may not be mutated while the Decoder remains in use.
If the decoder is currently at the end token for an object or array,
then it reports a [SyntacticError] and the internal state remains unchanged.
It returns [io.EOF] if there are no more values. Reset resets a decoder such that it is reading afresh from r and
configured with the provided options. Reset must not be called on
a Decoder passed to the [encoding/json/v2.UnmarshalerFrom.UnmarshalJSONFrom] method
or the [encoding/json/v2.UnmarshalFromFunc] function. SkipValue is semantically equivalent to calling [Decoder.ReadValue] and discarding
the result, except that memory is not wasted trying to hold the entire result. StackDepth returns the depth of the state machine for JSON data
that has already been read.
Each level on the stack represents a nested JSON object or array.
It is incremented whenever a [BeginObject] or [BeginArray] token is encountered
and decremented whenever an [EndObject] or [EndArray] token is encountered.
StackDepth returns 0 when not inside any object or array.
In particular, it returns 0 before any tokens have been read,
after any top-level value has been read, and between values
when decoding a stream of top-level values (e.g., NDJSON).
StackDepth returns 1 inside a top-level object or array,
2 inside a nested object or array, and so on.
For example, consider decoding the following JSON:
{"a": [1, 2], "b": {"c": 3}}
While decoding, StackDepth would report the following:
- At the start, StackDepth reports 0.
- After decoding the outer '{', StackDepth reports 1.
- After decoding the inner '[', StackDepth reports 2.
- After decoding the inner ']', StackDepth reports 1.
- After decoding the outer '}', StackDepth reports 0. StackIndex returns information about the specified stack level.
It must be a number between 0 and [Decoder.StackDepth], inclusive.
For each level, it reports the kind:
- [KindInvalid] for a level of zero,
- [KindBeginObject] for a level representing a JSON object, and
- [KindBeginArray] for a level representing a JSON array.
It also reports the length of that JSON object or array decoded so far.
Each name and value in a JSON object is counted separately,
so the effective number of members is half the length.
A complete JSON object must have an even length. StackPointer returns a JSON Pointer (RFC 6901) to the most recently read value. UnreadBuffer returns the data remaining in the unread buffer,
which may contain zero or more bytes.
This is the data already consumed from the input [io.Reader],
but not yet read by a [Decoder.ReadToken] or [Decoder.ReadValue] call.
It may contain bytes that do not form valid JSON, since it has not yet
been validated according to the JSON grammar.
The exact amount of buffered data is an implementation detail
of the Decoder and may change over time.
It is the caller's responsibility to concatenate this buffer with
the remainder of the input Reader to obtain the full sequence
of bytes after the last read JSON token or value.
The returned buffer must not be mutated while Decoder continues to be used.
The buffer contents are valid until the next Peek, Read, or Skip call.
*Decoder : encoding/json/v2.coder
func NewDecoder(r io.Reader, opts ...Options) *Decoder
func getBufferedDecoder(b []byte, opts ...Options) *Decoder
func getStreamingDecoder(r io.Reader, opts ...Options) *Decoder
func encoding/json.(*Number).UnmarshalJSONFrom(dec *Decoder) error
func encoding/json/v2.UnmarshalDecode(in *Decoder, out any, opts ...json.Options) (err error)
func encoding/json/v2.UnmarshalerFrom.UnmarshalJSONFrom(*Decoder) error
func mustReorderObjectsFromDecoder(d *Decoder, scratch *[]byte)
func putBufferedDecoder(d *Decoder)
func putStreamingDecoder(d *Decoder)
func encoding/json/v2.newUnmarshalErrorAfter(d *Decoder, t reflect.Type, err error) error
func encoding/json/v2.newUnmarshalErrorAfterWithSkipping(d *Decoder, t reflect.Type, err error) error
func encoding/json/v2.newUnmarshalErrorAfterWithValue(d *Decoder, t reflect.Type, err error) error
func encoding/json/v2.newUnmarshalErrorBefore(d *Decoder, t reflect.Type, err error) error
func encoding/json/v2.newUnmarshalErrorBeforeWithSkipping(d *Decoder, t reflect.Type, err error) error
func encoding/json/v2.unmarshalArrayAny(dec *Decoder, uo *jsonopts.Struct) ([]any, error)
func encoding/json/v2.unmarshalDecode(in *Decoder, out any, uo *jsonopts.Struct, last bool) (err error)
func encoding/json/v2.unmarshalEmbeddedFallbackNext(dec *Decoder, va json.addressableValue, uo *jsonopts.Struct, f *json.structField, quotedName, unquotedName []byte) error
func encoding/json/v2.unmarshalObjectAny(dec *Decoder, uo *jsonopts.Struct) (map[string]any, error)
func encoding/json/v2.unmarshalValueAny(dec *Decoder, uo *jsonopts.Struct) (any, error)
Encoder is a streaming encoder to raw JSON tokens and values.
It is used to write a stream of top-level JSON values,
each terminated with a newline character.
[Encoder.WriteToken] and [Encoder.WriteValue] calls may be interleaved.
For example, the following JSON value:
{"name":"value","array":[null,false,true,3.14159],"object":{"k":"v"}}
can be composed with the following calls (ignoring errors for brevity):
e.WriteToken(BeginObject) // {
e.WriteToken(String("name")) // "name"
e.WriteToken(String("value")) // "value"
e.WriteValue(Value(`"array"`)) // "array"
e.WriteToken(BeginArray) // [
e.WriteToken(Null) // null
e.WriteToken(False) // false
e.WriteValue(Value("true")) // true
e.WriteToken(Float(3.14159)) // 3.14159
e.WriteToken(EndArray) // ]
e.WriteValue(Value(`"object"`)) // "object"
e.WriteValue(Value(`{"k":"v"}`)) // {"k":"v"}
e.WriteToken(EndObject) // }
The above is one of many possible sequences of calls and
may not represent the most sensible method to call for any given token/value.
For example, it is probably more common to call [Encoder.WriteToken] with a string
for object names.sencoderState AvailableBuffer returns a zero-length buffer with a possible non-zero capacity.
This buffer is intended to be used to populate a [Value]
being passed to an immediately succeeding [Encoder.WriteValue] call.
Example usage:
b := e.AvailableBuffer()
b = append(b, '"')
b = appendString(b, v) // append the string formatting of v
b = append(b, '"')
... := e.WriteValue(b)
Note that WriteValue expects a valid JSON value. Constructing a value in a
raw []byte requires more care than using constructor functions like
[String], which always return valid [Token]s or [Value]s. Options returns the options used to construct the encoder and
may additionally contain semantic options passed to a
[encoding/json/v2.MarshalEncode] call.
If operating within
a [encoding/json/v2.MarshalerTo.MarshalJSONTo] method call or
a [encoding/json/v2.MarshalToFunc] function call,
then the returned options are only valid within the call. OutputOffset returns the current output byte offset. It gives the location
of the next byte immediately after the most recently written token or value.
The number of bytes actually written to the underlying [io.Writer] may be less
than this offset due to internal buffering. Reset resets an encoder such that it is writing afresh to w and
configured with the provided options. Reset must not be called on
an Encoder passed to the [encoding/json/v2.MarshalerTo.MarshalJSONTo] method
or the [encoding/json/v2.MarshalToFunc] function. StackDepth returns the depth of the state machine for written JSON data.
Each level on the stack represents a nested JSON object or array.
It is incremented whenever a [BeginObject] or [BeginArray] token is encountered
and decremented whenever an [EndObject] or [EndArray] token is encountered.
StackDepth returns 0 when not inside any object or array.
In particular, it returns 0 before any tokens have been written,
after any top-level value has been written, and between values
when encoding a stream of top-level values (e.g., NDJSON).
StackDepth returns 1 inside a top-level object or array,
2 inside a nested object or array, and so on.
For example, consider encoding the following JSON:
{"a": [1, 2], "b": {"c": 3}}
While encoding, StackDepth would report the following:
- At the start, StackDepth reports 0.
- After encoding the outer '{', StackDepth reports 1.
- After encoding the inner '[', StackDepth reports 2.
- After encoding the inner ']', StackDepth reports 1.
- After encoding the outer '}', StackDepth reports 0. StackIndex returns information about the specified stack level.
It must be a number between 0 and [Encoder.StackDepth], inclusive.
For each level, it reports the kind:
- [KindInvalid] for a level of zero,
- [KindBeginObject] for a level representing a JSON object, and
- [KindBeginArray] for a level representing a JSON array.
It also reports the length of that JSON object or array encoded so far.
Each name and value in a JSON object is counted separately,
so the effective number of members is half the length.
A complete JSON object must have an even length. StackPointer returns a JSON Pointer (RFC 6901) to the most recently written value. WriteToken writes the next token and advances the internal write offset.
The provided token kind must be consistent with the JSON grammar.
For example, it is an error to provide a number when the encoder
is expecting an object name (which is always a string), or
to provide an end object delimiter when the encoder is finishing an array.
If the provided token is invalid, then WriteToken reports a [SyntacticError] and
the internal state remains unchanged. The offset reported
in [SyntacticError] will be the [Encoder.OutputOffset] plus any delimiter
or whitespace characters that would have preceded the provided token. WriteValue writes the next raw value and advances the internal write offset.
The Encoder does not simply copy the provided value verbatim, but
parses it to ensure that it is syntactically valid and reformats it
according to how the Encoder is configured to format whitespace and strings.
If [AllowInvalidUTF8] is specified, then any invalid UTF-8 is mangled
as the Unicode replacement character, U+FFFD.
The provided value kind must be consistent with the JSON grammar
(see examples on [Encoder.WriteToken]). If the provided value is invalid,
then WriteValue reports a [SyntacticError] and the internal state remains unchanged.
The offset reported in [SyntacticError] will be the [Encoder.OutputOffset]
plus the offset into v of any encountered syntax error.
*Encoder : encoding/json/v2.coder
func NewEncoder(w io.Writer, opts ...Options) *Encoder
func getBufferedEncoder(opts ...Options) *Encoder
func getStreamingEncoder(w io.Writer, opts ...Options) *Encoder
func encoding/json.Number.MarshalJSONTo(enc *Encoder) error
func encoding/json/v2.MarshalEncode(out *Encoder, in any, opts ...json.Options) (err error)
func encoding/json/v2.MarshalerTo.MarshalJSONTo(*Encoder) error
func putBufferedEncoder(e *Encoder)
func putStreamingEncoder(e *Encoder)
func encoding/json/v2.marshalArrayAny(enc *Encoder, arr []any, mo *jsonopts.Struct) error
func encoding/json/v2.marshalEmbeddedFallbackAll(enc *Encoder, va json.addressableValue, mo *jsonopts.Struct, f *json.structField, insertUnquotedName func([]byte) bool) error
func encoding/json/v2.marshalEncode(out *Encoder, in any, mo *jsonopts.Struct) (err error)
func encoding/json/v2.marshalObjectAny(enc *Encoder, obj map[string]any, mo *jsonopts.Struct) error
func encoding/json/v2.marshalValueAny(enc *Encoder, val any, mo *jsonopts.Struct) error
func encoding/json/v2.newMarshalErrorBefore(e *Encoder, t reflect.Type, err error) error
Options configures [NewEncoder], [Encoder.Reset], [NewDecoder],
and [Decoder.Reset] with specific features.
Each function takes in a variadic list of options, where properties
set in later options override the value of previously set properties.
There is a single Options type, which is used with both encoding and decoding.
Some options affect both operations, while others only affect one operation:
- [AllowDuplicateNames] affects encoding and decoding
- [AllowInvalidUTF8] affects encoding and decoding
- [EscapeForHTML] affects encoding only
- [EscapeForJS] affects encoding only
- [PreserveRawStrings] affects encoding only
- [CanonicalizeRawInts] affects encoding only
- [CanonicalizeRawFloats] affects encoding only
- [ReorderRawObjects] affects encoding only
- [SpaceAfterColon] affects encoding only
- [SpaceAfterComma] affects encoding only
- [Multiline] affects encoding only
- [WithIndent] affects encoding only
- [WithIndentPrefix] affects encoding only
Options that do not affect a particular operation are ignored.
The Options type is identical to [encoding/json.Options] and
[encoding/json/v2.Options]. Options from the other packages may
be passed to functionality in this package, but are ignored.
Options from this package may be used with the other packages.
Pointer is a JSON Pointer (RFC 6901) that references a particular JSON value
relative to the root of the top-level JSON value.
A Pointer is a slash-separated list of tokens, where each token is
either a JSON object name or an index to a JSON array element
encoded as a base-10 integer value.
It is impossible to distinguish between an array index and an object name
(that happens to be a base-10 encoded integer) without also knowing
the structure of the top-level JSON value that the pointer refers to.
There is exactly one representation of a pointer to a particular value,
so comparability of Pointer values is equivalent to checking whether
they both point to the same value. AppendToken appends a token to the end of p and returns the full pointer. Contains reports whether the JSON value that p points to
is equal to or contains the JSON value that pc points to. IsValid reports whether p is a valid JSON Pointer according to RFC 6901.
Note that the concatenation of two valid pointers produces a valid pointer. LastToken returns the last token in the pointer.
The last token of an empty Pointer is the empty string. Parent strips off the last token and returns the remaining pointer.
The parent of an empty Pointer is the empty string. Tokens returns an iterator over the reference tokens in the JSON pointer,
from first to last.
Pointer : database/sql/driver.Validator
func (*Decoder).StackPointer() Pointer
func (*Encoder).StackPointer() Pointer
func Pointer.AppendToken(tok string) Pointer
func Pointer.Parent() Pointer
func Pointer.Contains(pc Pointer) bool
func encoding/json/v2.newDuplicateNameError(ptr Pointer, quotedName []byte, offset int64) error
SyntacticError is a description of an error that occurred when
encoding or decoding JSON according to the grammar.
The contents of this error as produced by this package may change over time. ByteOffset indicates that an error occurred at or after this byte offset. Err is the underlying error. JSONPointer indicates that an error occurred within this JSON value
as indicated using the JSON Pointer notation (see RFC 6901).nonComparablenonComparablerequireKeyedLiteralsrequireKeyedLiterals(*SyntacticError) Error() string(*SyntacticError) Unwrap() error
*SyntacticError : error
Token represents a lexical JSON token, which may be one of the following:
- a JSON literal (i.e., null, true, or false)
- a JSON string (e.g., "hello, world!")
- a JSON number (e.g., 123.456)
- a begin or end delimiter for a JSON object (i.e., { or } )
- a begin or end delimiter for a JSON array (i.e., [ or ] )
A Token cannot represent entire array or object values, while a [Value] can.
There is no Token to represent commas and colons since
these structural tokens can be inferred from the surrounding context.
A Token stores data in one of two forms:
- As raw JSON text: backed by the internal buffer of the [Decoder]
and only ever produced by [Decoder.ReadToken].
Such a token is only valid until the next call to any method on that
[Decoder] (e.g., [Decoder.PeekKind], [Decoder.ReadToken],
[Decoder.ReadValue], or [Decoder.SkipValue]).
Call [Token.Clone] to copy the raw text into an independent allocation
that persists beyond subsequent [Decoder] calls.
- As a typed Go value: a self-contained representation produced by
the constructor functions (e.g., [String], [Int], [Uint], [Float]).
Such tokens are valid indefinitely and do not need to be cloned.nonComparablenonComparable num is a float32, float64, int64, or uint64 stored as a uint64 value.
For floating-point values, it stores the raw IEEE-754 bit-pattern.
It is non-zero for any JSON number in the "exact" form. raw contains a reference to the raw decode buffer.
If non-nil, then its value takes precedence over str and num.
It is only valid if num == raw.previousOffsetStart(). str is the unescaped JSON string if num is zero.
Otherwise, it is "F", "f", "i", or "u" if num should be interpreted
as a float32, float64, int64, or uint64, respectively. Bool returns the value for a JSON boolean.
It panics if the token kind is not a JSON boolean. Clone returns a copy of the token with a value that is not backed by the
[Decoder] buffer and therefore remains valid past subsequent [Decoder] calls.
It has no effect on tokens produced by constructor functions,
since those are already self-contained. Float returns the floating-point value for a JSON number
parsed according to 64 bits of precision.
If the JSON number is outside the representable range of a float64,
it returns +Inf or -Inf along with an error
that matches [strconv.ErrRange] according to [errors.Is].
It returns a NaN, +Inf, or -Inf value for any JSON string
with the values "NaN", "Infinity", or "-Infinity".
It panics if the token kind is not a JSON number
or a JSON string with the aforementioned values. Float32 returns the floating-point value for a JSON number
parsed according to 32 bits of precision.
If the JSON number is outside the representable range of a float32,
it returns +Inf or -Inf along with an error
that matches [strconv.ErrRange] according to [errors.Is].
It returns a NaN, +Inf, or -Inf value for any JSON string
with the values "NaN", "Infinity", or "-Infinity".
It panics if the token kind is not a JSON number
or a JSON string with the aforementioned values.
Note that most JSON libraries and standards assume that JSON numbers
are 64-bit floating-point numbers.
This method should only be used if the caller knows
from other context that this token is a JSON number
formatted only to 32 bits of precision (such as being encoded
using the [Float32] constructor). For all other situations,
prefer using the [Token.Float] accessor instead. Int returns the signed integer value for a JSON number.
It reports an error that matches [strconv.ErrSyntax] according to [errors.Is]
if the JSON number does not match the restricted grammar of just a signed integer.
It reports an error that matches [strconv.ErrRange] according to [errors.Is]
if the JSON number is a signed integer, but outside the range of an int64.
Even if an error is reported, a reasonable value is still returned.
The fractional component of any number is ignored (truncation toward zero).
Any number beyond the representation of an int64 will be saturated
to the closest representable value.
It panics if the token kind is not a JSON number. Kind returns the token kind. String returns the unescaped string value for a JSON string.
For other JSON kinds, this returns the raw JSON representation. Uint returns the unsigned integer value for a JSON number.
It reports an error that matches [strconv.ErrSyntax] if the JSON number
does not match the restricted grammar of just an unsigned integer.
It reports an error that matches [strconv.ErrRange] if the JSON number
is an unsigned integer, but outside the representable range of a uint64.
Even if an error is reported, a reasonable value is still returned.
The fractional component of any number is ignored (truncation toward zero).
Any number beyond the representation of a uint64 will be saturated
to the closest representable value.
It panics if the token kind is not a JSON number. appendNumber appends a JSON number to dst and returns it.
It panics if t is not a JSON number. appendString appends a JSON string to dst and returns it.
It panics if t is not a JSON string.( Token) float(bits int) (float64, error)( Token) string() (string, []byte)
Token : expvar.Var
Token : fmt.Stringer
Token : context.stringer
Token : runtime.stringer
func Bool(b bool) Token
func Float(n float64) Token
func Float32(n float32) Token
func Int(n int64) Token
func String(s string) Token
func Uint(n uint64) Token
func (*Decoder).ReadToken() (Token, error)
func Token.Clone() Token
func rawToken(s string) Token
func (*Encoder).WriteToken(t Token) error
var BeginArray
var BeginObject
var EndArray
var EndObject
var False
var Null
var True
var nanString
var ninfString
var pinfString
var zeroNumber
var zeroString
Value represents a single raw JSON value, which may be one of the following:
- a JSON literal (i.e., null, true, or false)
- a JSON string (e.g., "hello, world!")
- a JSON number (e.g., 123.456)
- an entire JSON object (e.g., {"fizz":"buzz"} )
- an entire JSON array (e.g., [1,2,3] )
Value can represent entire array or object values, while [Token] cannot.
Value may contain leading and/or trailing whitespace. Canonicalize canonicalizes the raw JSON value according to the
JSON Canonicalization Scheme (JCS) as defined by RFC 8785.
Canonicalization produces a JSON value with the same meaning as the original,
but is stable in the sense that calling Canonicalize on a canonicalized
value does nothing.
JSON strings are formatted to use their minimal representation,
JSON numbers are formatted as double precision numbers according
to some stable serialization algorithm.
JSON object members are sorted in ascending order by name.
All whitespace is removed.
Canonicalize is equivalent to calling [Value.Format] with the following options:
- [CanonicalizeRawInts](true)
- [CanonicalizeRawFloats](true)
- [ReorderRawObjects](true)
Any options specified by the caller are applied after the initial set
and may deliberately override prior options.
Note that JCS treats all JSON numbers as IEEE 754 double precision numbers.
Any numbers with precision beyond what is representable by that form
will lose their precision when canonicalized. For example, integer values
beyond ±2⁵³ will lose their precision. To preserve the original representation
of JSON integers, additionally set [CanonicalizeRawInts] to false:
v.Canonicalize(jsontext.CanonicalizeRawInts(false)) Clone returns a copy of v. Compact removes all whitespace from the raw JSON value.
It does not reformat JSON strings or numbers to use any other representation.
To maximize the set of JSON values that can be formatted,
it permits values with duplicate names and invalid UTF-8.
Compact is equivalent to calling [Value.Format] with the following options:
- [AllowDuplicateNames](true)
- [AllowInvalidUTF8](true)
- [PreserveRawStrings](true)
Any options specified by the caller are applied after the initial set
and may deliberately override prior options. Format formats the raw JSON value in place.
By default (if no options are specified), it validates according to RFC 7493
and produces the minimal JSON representation, where
all whitespace is elided and JSON strings use the shortest encoding.
Relevant options include:
- [AllowDuplicateNames]
- [AllowInvalidUTF8]
- [EscapeForHTML]
- [EscapeForJS]
- [PreserveRawStrings]
- [CanonicalizeRawInts]
- [CanonicalizeRawFloats]
- [ReorderRawObjects]
- [SpaceAfterColon]
- [SpaceAfterComma]
- [Multiline]
- [WithIndent]
- [WithIndentPrefix]
All other options are ignored.
It is guaranteed to succeed if the value is valid according to the same options.
If the value is already formatted, then the buffer is not mutated. Indent reformats the whitespace in the raw JSON value so that each element
in a JSON object or array begins on an indented line according to the nesting.
It does not reformat JSON strings or numbers to use any other representation.
To maximize the set of JSON values that can be formatted,
it permits values with duplicate names and invalid UTF-8.
Indent is equivalent to calling [Value.Format] with the following options:
- [AllowDuplicateNames](true)
- [AllowInvalidUTF8](true)
- [PreserveRawStrings](true)
- [Multiline](true)
Any options specified by the caller are applied after the initial set
and may deliberately override prior options. IsValid reports whether the raw JSON value is syntactically valid
according to the specified options.
By default (if no options are specified), it validates according to RFC 7493.
It verifies whether the input is properly encoded as UTF-8,
that escape sequences within strings decode to valid Unicode codepoints, and
that all names in each object are unique.
It does not verify whether numbers are representable within the limits
of any common numeric type (e.g., float64, int64, or uint64).
Relevant options include:
- [AllowDuplicateNames]
- [AllowInvalidUTF8]
All other options are ignored. Kind returns the starting token kind.
For a valid value, this will never include [KindEndObject] or [KindEndArray]. MarshalJSON returns v as the JSON encoding of v.
It performs no validation.
If v is nil, then this returns a JSON null. String returns the string formatting of v. UnmarshalJSON sets v as the JSON encoding of b.
It stores a copy of the provided raw JSON input without any validation. format accepts two []Options to avoid the allocation of appending them together.
It is equivalent to v.Format(append(opts1, opts2...)...).
Value : encoding/json/v2.Marshaler
*Value : encoding/json/v2.Unmarshaler
Value : expvar.Var
Value : fmt.Stringer
Value : context.stringer
Value : runtime.stringer
func (*Decoder).ReadValue() (Value, error)
func Value.Clone() Value
func (*Encoder).WriteValue(v Value) error
bufferStatistics is statistics to track buffer utilization.
It is used to determine whether to recycle a buffer or not
to avoid https://go.dev/issue/23199. // length of previous buffer // number of times the buffer was under-utilized
decodeBuffer is a buffer split into 4 segments:
- buf[0:prevEnd] // already read portion of the buffer
- buf[prevStart:prevEnd] // previously read value
- buf[prevEnd:len(buf)] // unread portion of the buffer
- buf[len(buf):cap(buf)] // unused portion of the buffer
Invariants:
0 ≤ prevStart ≤ prevEnd ≤ len(buf) ≤ cap(buf) baseOffset is added to prevStart and prevEnd to obtain
the absolute offset relative to the start of io.Reader stream. // may alias rd if it is a bytes.Buffer // implies peekPos is -1 // non-zero if valid offset into buf for start of next tokenprevEndintprevStartintrdio.Reader PreviousTokenOrValue returns the previously read token or value
unless it has been invalidated by a call to PeekKind.
If a token is just a delimiter, then this returns a 1-byte buffer.
This method is used for error reporting at the semantic layer. invalidatePreviousRead invalidates buffers returned by Peek and Read calls
so that the first byte is an invalid character.
This Hyrum-proofs the API against faulty application code that assumes
values returned by ReadValue remain valid past subsequent Read calls. needMore reports whether there are no more unread bytes.(*decodeBuffer) offsetAt(pos int) int64(*decodeBuffer) previousBuffer() []byte(*decodeBuffer) previousOffsetEnd() int64(*decodeBuffer) previousOffsetStart() int64(*decodeBuffer) unreadBuffer() []byte
decoderState is the low-level state of Decoder.
It has exported fields and methods for use by the "json" package. Names is a stack of object names. Namespaces is a stack of object namespaces.
For performance reasons, Encoder or Decoder may not update this
if Marshal or Unmarshal is able to track names in a more efficient way.
See makeMapArshaler and makeStructArshaler.
Not used if AllowDuplicateNames is true. Tokens validates whether the next token kind is valid. // only used when unmarshaling; identical to json.stringCacheStructjsonopts.StructStruct.ArshalValuesjsonopts.ArshalValues // valid if jsonflags.FormatTag is set // jsonflags.Marshalers // jsonflags.UnmarshalersStruct.CoderValuesjsonopts.CoderValues // jsonflags.ByteLimit // jsonflags.DepthLimit // jsonflags.Indent // jsonflags.IndentPrefixStruct.Flagsjsonflags.FlagsdecodeBufferdecodeBuffer baseOffset is added to prevStart and prevEnd to obtain
the absolute offset relative to the start of io.Reader stream. // may alias rd if it is a bytes.Buffer // implies peekPos is -1 // non-zero if valid offset into buf for start of next tokendecodeBuffer.prevEndintdecodeBuffer.prevStartintdecodeBuffer.rdio.Readerstatestate(*decoderState) AppendStackPointer(b []byte, where int) []byte AtEOF reports whether the decoder is at EOF. CheckEOF verifies that the input has no more data. CheckNextValue checks whether the next value is syntactically valid,
but does not advance the read offset.
If last, it verifies that the stream cleanly terminates with [io.EOF]. CountNextDelimWhitespace counts the number of upcoming bytes of
delimiter or whitespace characters.
This method is used for error reporting at the semantic layer. InitializeMultiline sets default options implied by Multiline.(*decoderState) JSONOptions(internal.NotForPublicUse)(*decoderState) Join(srcs ...jsonopts.Options)(*decoderState) PeekKind() Kind PreviousTokenOrValue returns the previously read token or value
unless it has been invalidated by a call to PeekKind.
If a token is just a delimiter, then this returns a 1-byte buffer.
This method is used for error reporting at the semantic layer.(*decoderState) ReadToken() (Token, error)(*decoderState) ReadValue(flags *jsonwire.ValueFlags) (Value, error) SkipUntil skips all tokens until the state machine
is at or past the specified depth and length.(*decoderState) SkipValue() error SkipValueRemainder skips the remainder of a value
after reading a '{' or '[' token. appendStackPointer appends a JSON Pointer (RFC 6901) to the current value.
- If where is -1, then it points to the previously processed token.
- If where is 0, then it points to the parent JSON object or array,
or an object member if in-between an object member key and value.
This is useful when the position is ambiguous whether
we are interested in the previous or next token, or
when we are uncertain whether the next token
continues or terminates the current object or array.
- If where is +1, then it points to the next expected value,
assuming that it continues the current JSON object or array.
As a special case, if the next token is a JSON object name,
then it points to the parent JSON object.
Invariant: Must call s.names.copyQuotedBuffer beforehand. checkDelim checks whether delim is valid for the given next kind. checkDelimBeforeIOError checks whether the delim is even valid
before returning an IO error, which occurs after the delim.(*decoderState) checkEOF(pos int) error consumeArray consumes a single JSON array starting at d.buf[pos:].
It returns the new position in d.buf immediately after the array. consumeLiteral consumes a single JSON literal starting at d.buf[pos:].
It returns the new position in d.buf immediately after the literal. consumeNumber consumes a single JSON number starting at d.buf[pos:].
It returns the new position in d.buf immediately after the number. consumeObject consumes a single JSON object starting at d.buf[pos:].
It returns the new position in d.buf immediately after the object. consumeString consumes a single JSON string starting at d.buf[pos:].
It returns the new position in d.buf immediately after the string. consumeValue consumes a single JSON value starting at d.buf[pos:].
It returns the new position in d.buf immediately after the value. consumeWhitespace consumes all whitespace starting at d.buf[pos:].
It returns the new position in d.buf immediately after the last whitespace.
If it returns nil, there is guaranteed to at least be one unread byte.
The following pattern is common in this implementation:
pos += jsonwire.ConsumeWhitespace(d.buf[pos:])
if d.needMore(pos) {
if pos, err = d.consumeWhitespace(pos); err != nil {
return ...
}
}
It is difficult to simplify this without sacrificing performance since
consumeWhitespace must be inlined. The body of the if statement is
executed only in rare situations where we need to fetch more data.
Since fetching may return an error, we also need to check the error. fetch reads at least 1 byte from the underlying io.Reader.
It returns io.ErrUnexpectedEOF if zero bytes were read and io.EOF was seen. invalidatePreviousRead invalidates buffers returned by Peek and Read calls
so that the first byte is an invalid character.
This Hyrum-proofs the API against faulty application code that assumes
values returned by ReadValue remain valid past subsequent Read calls. needMore reports whether there are no more unread bytes. needObjectValue reports whether the next token should be an object value.
This method is used by [wrapSyntacticError].(*decoderState) offsetAt(pos int) int64(*decoderState) options() *jsonopts.Struct(*decoderState) previousBuffer() []byte(*decoderState) previousOffsetEnd() int64(*decoderState) previousOffsetStart() int64(*decoderState) reset(b []byte, r io.Reader, opts ...Options)(*decoderState) unreadBuffer() []byte
*decoderState : encoding/json/internal/jsonopts.Options
encodeBuffer is a buffer split into 2 segments:
- buf[0:len(buf)] // written (but unflushed) portion of the buffer
- buf[len(buf):cap(buf)] // unused portion of the buffer // may alias wr if it is a bytes.Buffer availBuffer is the buffer returned by the AvailableBuffer method. // always has zero length baseOffset is added to len(buf) to obtain the absolute offset
relative to the start of io.Writer stream. bufStats is statistics about buffer utilization.
It is only used with pooled encoders in pools.go. maxValue is the approximate maximum Value size passed to WriteValue.wrio.Writer(*encodeBuffer) offsetAt(pos int) int64(*encodeBuffer) previousOffsetEnd() int64(*encodeBuffer) unflushedBuffer() []byte
encoderState is the low-level state of Encoder.
It has exported fields and methods for use by the "json" package. // may alias wr if it is a bytes.Buffer // only used when marshaling; identical to json.seenPointers Names is a stack of object names. Namespaces is a stack of object namespaces.
For performance reasons, Encoder or Decoder may not update this
if Marshal or Unmarshal is able to track names in a more efficient way.
See makeMapArshaler and makeStructArshaler.
Not used if AllowDuplicateNames is true. Tokens validates whether the next token kind is valid.Structjsonopts.StructStruct.ArshalValuesjsonopts.ArshalValues // valid if jsonflags.FormatTag is set // jsonflags.Marshalers // jsonflags.UnmarshalersStruct.CoderValuesjsonopts.CoderValues // jsonflags.ByteLimit // jsonflags.DepthLimit // jsonflags.Indent // jsonflags.IndentPrefixStruct.Flagsjsonflags.FlagsencodeBufferencodeBuffer availBuffer is the buffer returned by the AvailableBuffer method. // always has zero length baseOffset is added to len(buf) to obtain the absolute offset
relative to the start of io.Writer stream. bufStats is statistics about buffer utilization.
It is only used with pooled encoders in pools.go. maxValue is the approximate maximum Value size passed to WriteValue.encodeBuffer.wrio.Writerstatestate AppendIndent appends the appropriate number of indentation characters
for the current nested level, n. AppendRaw appends either a raw string (without double quotes) or number.
Specify safeASCII if the string output is guaranteed to be ASCII
without any characters (including '<', '>', and '&') that need escaping,
otherwise this will validate whether the string needs escaping.
The appended bytes for a JSON number must be valid.
This is a specialized implementation of Encoder.WriteValue
that allows appending directly into the buffer.
It is only called from marshal logic in the "json" package.(*encoderState) AppendStackPointer(b []byte, where int) []byte CountNextDelimWhitespace counts the number of bytes of delimiter and
whitespace bytes assuming the upcoming token is a JSON value.
This method is used for error reporting at the semantic layer. Flush flushes the buffer to the underlying io.Writer.
It may append a trailing newline after the top-level value. InitializeMultiline sets default options implied by Multiline.(*encoderState) JSONOptions(internal.NotForPublicUse)(*encoderState) Join(srcs ...jsonopts.Options) NeedFlush determines whether to flush at this point. UnwriteEmptyObjectMember unwrites the last object member if it is empty
and reports whether it performed an unwrite operation. UnwriteOnlyObjectMemberName unwrites the only object member name
and returns the unquoted name.(*encoderState) WriteToken(t Token) error(*encoderState) WriteValue(v Value) error appendStackPointer appends a JSON Pointer (RFC 6901) to the current value.
- If where is -1, then it points to the previously processed token.
- If where is 0, then it points to the parent JSON object or array,
or an object member if in-between an object member key and value.
This is useful when the position is ambiguous whether
we are interested in the previous or next token, or
when we are uncertain whether the next token
continues or terminates the current object or array.
- If where is +1, then it points to the next expected value,
assuming that it continues the current JSON object or array.
As a special case, if the next token is a JSON object name,
then it points to the parent JSON object.
Invariant: Must call s.names.copyQuotedBuffer beforehand. appendWhitespace appends whitespace that immediately precedes the next token. avoidFlush indicates whether to avoid flushing to ensure there is always
enough in the buffer to unwrite the last object member if it were empty. needObjectValue reports whether the next token should be an object value.
This method is used by [wrapSyntacticError].(*encoderState) offsetAt(pos int) int64(*encoderState) options() *jsonopts.Struct(*encoderState) previousOffsetEnd() int64 reformatArray parses a JSON array from the start of src and
appends it to the end of dst, reformatting whitespace and strings as needed.
It returns the extended dst buffer and the number of consumed input bytes. reformatObject parses a JSON object from the start of src and
appends it to the end of dst, reformatting whitespace and strings as needed.
It returns the extended dst buffer and the number of consumed input bytes. reformatValue parses a JSON value from the start of src and
appends it to the end of dst, reformatting whitespace and strings as needed.
It returns the extended dst buffer and the number of consumed input bytes.(*encoderState) reset(b []byte, w io.Writer, opts ...Options)(*encoderState) unflushedBuffer() []byte
*encoderState : encoding/json/internal/jsonopts.Options
Export exposes internal functionality from "jsontext" to "json".
This cannot be dynamically called by other packages since
they cannot obtain a reference to the internal.AllowInternalUse value.
var Internal
nonComparable can be embedded in a struct to prevent comparability.
// either "Int", "Uint", or "Float" // either [strconv.ErrSyntax] or [strconv.ErrRange] // e.g., "1e1000"(*numError) Error() string(*numError) Unwrap() error
*numError : error
buffer is the entirety of the raw JSON object member
starting from right after the previous member (or opening '{')
until right after the member value. // e.g., `, \n\r\t"name": "value"` name is the unquoted name. // e.g., "name"( objectMember) Compare(y objectMember) int
func getObjectMembers() *[]objectMember
func putObjectMembers(ns *[]objectMember)
objectNamespace is the namespace for a JSON object.
In contrast to objectNameStack, this needs to remember all names
per JSON object.
The zero value is an empty namespace ready for use. allUnquotedNames is a back-to-back concatenation of every name in the namespace. endOffsets is a list of offsets to the end of each name in buffers.
The length of offsets is the number of names in the namespace. mapNames is a Go map containing every name in the namespace.
Only valid if non-nil.(*objectNamespace) InsertUnquoted(name []byte) bool getUnquoted retrieves the ith unquoted name in the namespace.(*objectNamespace) insert(name []byte, quoted bool) bool insertQuoted inserts a name and reports whether it was inserted,
which only occurs if name is not already in the namespace.
The provided name must be a valid JSON string. lastUnquoted retrieves the last name in the namespace. length reports the number of names in the namespace. removeLast removes the last name in the namespace. reset resets the namespace to be empty.
objectNamespaceStack is a stack of object namespaces.
This data structure assists in detecting duplicate names. Last returns a pointer to the last JSON object namespace. pop terminates the namespace for a nested JSON object. push starts a new namespace for a nested JSON object. reset resets the object namespace stack.
objectNameStack is a stack of names when descending into a JSON object.
In contrast to objectNamespaceStack, this only has to remember a single name
per JSON object.
This data structure may contain offsets to encodeBuffer or decodeBuffer.
It violates clean abstraction of layers, but is significantly more efficient.
This ensures that popping and pushing in the common case is a trivial
push/pop of an offset integer.
The zero value is an empty names stack ready for use. offsets is a stack of offsets for each name.
A non-negative offset is the ending offset into the local names buffer.
A negative offset is the bit-wise inverse of a starting offset into
a remote buffer (e.g., encodeBuffer or decodeBuffer).
A math.MinInt offset at the end implies that the last object is empty.
Invariant: Positive offsets always occur before negative offsets. unquotedNames is a back-to-back concatenation of names. ReplaceLastQuotedOffset replaces the last name with the starting offset
to the quoted name in some remote buffer. All offsets provided must be
relative to the same buffer until copyQuotedBuffer is called. clearLast removes any name in the last JSON object.
It is semantically equivalent to ns.push followed by ns.pop. copyQuotedBuffer copies names from the remote buffer into the local names
buffer so that there are no more offset references into the remote buffer.
This allows the remote buffer to change contents without affecting
the names that this data structure is trying to remember.(*objectNameStack) ensureCopiedBuffer() getUnquoted retrieves the ith unquoted name in the stack.
It returns an empty string if the last object is empty.
Invariant: Must call copyQuotedBuffer beforehand.(*objectNameStack) length() int pop ascends out of a nested JSON object. push descends into a nested JSON object. replaceLastUnquotedName replaces the last name with the provided name.
Invariant: Must call copyQuotedBuffer beforehand.(*objectNameStack) reset()
pointerSuffixError represents a JSON pointer suffix to be appended
to [SyntacticError.JSONPointer]. It is an internal error type
used within this package and does not appear in the public API.
This type is primarily used to annotate errors in Encoder.WriteValue
and Decoder.ReadValue with precise positions.
At the time WriteValue or ReadValue is called, a JSON pointer to the
upcoming value can be constructed using the Encoder/Decoder state.
However, tracking pointers within values during normal operation
would incur a performance penalty in the error-free case.
To provide precise error locations without this overhead,
the error is wrapped with object names or array indices
as the call stack is popped when an error occurs.
Since this happens in reverse order, pointerSuffixError holds
the pointer in reverse and is only later reversed when appending to
the pointer prefix.
For example, if the encoder is at "/alpha/bravo/charlie"
and an error occurs in WriteValue at "/xray/yankee/zulu", then
the final pointer should be "/alpha/bravo/charlie/xray/yankee/zulu".
As pointerSuffixError is populated during the error return path,
it first contains "/zulu", then "/zulu/yankee",
and finally "/zulu/yankee/xray".
These tokens are reversed and concatenated to "/alpha/bravo/charlie"
to form the full pointer.errorerror reversePointer is a JSON pointer, but with each token in reverse order.( pointerSuffixError) Error() builtin.string appendPointer appends the path encoded in e to the end of pointer.
pointerSuffixError : error
requireKeyedLiterals can be embedded in a struct to require keyed literals.
Names is a stack of object names. Namespaces is a stack of object namespaces.
For performance reasons, Encoder or Decoder may not update this
if Marshal or Unmarshal is able to track names in a more efficient way.
See makeMapArshaler and makeStructArshaler.
Not used if AllowDuplicateNames is true. Tokens validates whether the next token kind is valid. appendStackPointer appends a JSON Pointer (RFC 6901) to the current value.
- If where is -1, then it points to the previously processed token.
- If where is 0, then it points to the parent JSON object or array,
or an object member if in-between an object member key and value.
This is useful when the position is ambiguous whether
we are interested in the previous or next token, or
when we are uncertain whether the next token
continues or terminates the current object or array.
- If where is +1, then it points to the next expected value,
assuming that it continues the current JSON object or array.
As a special case, if the next token is a JSON object name,
then it points to the parent JSON object.
Invariant: Must call s.names.copyQuotedBuffer beforehand. needObjectValue reports whether the next token should be an object value.
This method is used by [wrapSyntacticError].(*state) reset()
stateEntry encodes several artifacts within a single unsigned integer:
- whether this represents a JSON object or array,
- whether this object should check for duplicate names, and
- how many elements are in this JSON object or array. DisableNamespace disables the JSON object namespace such that the
Encoder or Decoder no longer updates the namespace. Increment increments the number of elements for the current object or array.
This assumes that overflow won't practically be an issue since
1<<bits.OnesCount(stateCountMask) is sufficiently large. Length reports the number of elements in the JSON object or array.
Each name and value in an object entry is treated as a separate element. NeedObjectName reports whether the next token must be a JSON string,
which is necessary for JSON object names. decrement decrements the number of elements for the current object or array.
It is the caller's responsibility to ensure that e.length > 0. invalidateNamespace marks the JSON object namespace as being invalid. isActiveNamespace reports whether the JSON object namespace is actively
being updated and used for duplicate name checks. isArray reports whether this is a JSON array. isObject reports whether this is a JSON object. isValidNamespace reports whether the JSON object namespace is valid. needImplicitColon reports whether a colon should occur next,
which always occurs after JSON object names. needImplicitComma reports whether a comma should occur next,
which always occurs after a value in a JSON object or array
before the next value (or name). needObjectValue reports whether the next token must be a JSON value,
which is necessary after every JSON object name.
const stateCountEven
const stateCountLSBMask
const stateCountMask
const stateCountOdd
const stateDisableNamespace
const stateInvalidNamespace
const stateNamespaceMask
const stateTypeArray
const stateTypeMask
const stateTypeObject
stateMachine is a push-down automaton that validates whether
a sequence of tokens is valid or not according to the JSON grammar.
It is useful for both encoding and decoding.
It is a stack where each entry represents a nested JSON object or array.
The stack has a minimum depth of 1 where the first level is a
virtual JSON array to handle a stream of top-level JSON values.
The top-level virtual JSON array is special in that it doesn't require commas
between each JSON value.
For performance, most methods are carefully written to be inlinable.
The zero value is a valid state machine ready for use.LaststateEntryStack[]stateEntry Depth is the current nested depth of JSON objects and arrays.
It is one-indexed (i.e., top-level values have a depth of 1). DepthLength reports the current nested depth and
the length of the last JSON object or array. InvalidateDisabledNamespaces marks all disabled namespaces as invalid.
For efficiency, Marshal and Unmarshal may disable namespaces since there are
more efficient ways to track duplicate names. However, if an error occurs,
the namespaces in Encoder or Decoder will be left in an inconsistent state.
Mark the namespaces as invalid so that future method calls on
Encoder or Decoder will return an error. MayAppendDelim appends a colon or comma that may precede the next token. NeedIndent reports whether indent whitespace should be injected.
A zero value means that no whitespace should be injected.
A positive value means '\n', indentPrefix, and (n-1) copies of indentBody
should be appended to the output immediately before the next token. appendLiteral appends a JSON literal as the next token in the sequence.
If an error is returned, the state is not mutated. appendNumber appends a JSON number as the next token in the sequence.
If an error is returned, the state is not mutated. appendString appends a JSON string as the next token in the sequence.
If an error is returned, the state is not mutated. index returns a reference to the ith entry.
It is only valid until the next push method call. needDelim reports whether a colon or comma token should be implicitly emitted
before the next token of the specified kind.
A zero value means no delimiter should be emitted. popArray appends a JSON end array token as next in the sequence.
If an error is returned, the state is not mutated. popObject appends a JSON end object token as next in the sequence.
If an error is returned, the state is not mutated. pushArray appends a JSON begin array token as next in the sequence.
If an error is returned, the state is not mutated. pushObject appends a JSON begin object token as next in the sequence.
If an error is returned, the state is not mutated. reset resets the state machine.
The machine always starts with a minimum depth of 1.
Package-Level Functions (total 45, in which 25 are exported)
AllowDuplicateNames specifies that JSON objects may contain
duplicate member names. Disabling the duplicate name check may provide
performance benefits, but breaks compliance with RFC 7493, section 2.3.
The input or output will still be compliant with RFC 8259,
which leaves the handling of duplicate names as unspecified behavior.
This affects either encoding or decoding.
AllowInvalidUTF8 specifies that JSON strings may contain invalid UTF-8,
which will be mangled as the Unicode replacement character, U+FFFD.
This causes the encoder or decoder to break compliance with
RFC 7493, section 2.1, and RFC 8259, section 8.1.
This affects either encoding or decoding.
AppendFloat appends src to dst as a JSON number per RFC 8259, section 6.
Except for -0, which is formatted as -0 instead of 0,
the output is identical to ECMA-262, 10th edition, section 7.1.12.1
and (for 64-bit precision) identical to RFC 8785, section 3.2.2.3.
The values NaN, +Inf, and -Inf will be represented as a JSON string
with the values "NaN", "Infinity", and "-Infinity".
Note that most JSON libraries and standards assume that JSON numbers
are 64-bit floating-point numbers. As such, prefer using 64 bits
of precision unless the recipient can know from other context
that the encoded number uses 32 bits of precision.
Type Parameters:
Bytes: ~[]byte | ~string AppendFormat formats the JSON value in src and appends it to dst
according to the specified options.
See [Value.Format] for more details about the formatting behavior.
The dst and src may overlap.
If an error is reported, then the entirety of src is appended to dst.
Type Parameters:
Bytes: ~[]byte | ~string AppendQuote appends a double-quoted JSON string literal representing src
to dst and returns the extended buffer.
It uses the minimal string representation per RFC 8785, section 3.2.2.2.
Invalid UTF-8 bytes are replaced with the Unicode replacement character
and an error is returned at the end indicating the presence of invalid UTF-8.
The dst must not overlap with the src.
Type Parameters:
Bytes: ~[]byte | ~string AppendUnquote appends the decoded interpretation of src as a
double-quoted JSON string literal to dst and returns the extended buffer.
The input src must be a JSON string without any surrounding whitespace.
Invalid UTF-8 bytes are replaced with the Unicode replacement character
and an error is returned at the end indicating the presence of invalid UTF-8.
Any trailing bytes after the JSON string literal results in an error.
The dst must not overlap with the src.
Bool constructs a Token representing a JSON boolean.
CanonicalizeRawFloats specifies that when encoding a raw JSON
floating-point number (i.e., a number with a fraction or exponent) in a
[Token] or [Value], the number is canonicalized
according to RFC 8785, section 3.2.2.3. As a special case,
the number -0 is canonicalized as 0.
JSON numbers are treated as IEEE 754 double precision numbers.
It is safe to canonicalize a serialized single precision number and
parse it back as a single precision number and expect the same value.
If a number exceeds ±1.7976931348623157e+308, which is the maximum
finite number, then it is saturated at that value and formatted as such.
This only affects encoding and is ignored when decoding.
CanonicalizeRawInts specifies that when encoding a raw JSON
integer number (i.e., a number without a fraction and exponent) in a
[Token] or [Value], the number is canonicalized
according to RFC 8785, section 3.2.2.3. As a special case,
the number -0 is canonicalized as 0.
JSON numbers are treated as IEEE 754 double precision numbers.
Any numbers with precision beyond what is representable by that form
will lose their precision when canonicalized. For example,
integer values beyond ±2⁵³ will lose their precision.
For example, 1234567890123456789 is formatted as 1234567890123456800.
This only affects encoding and is ignored when decoding.
EscapeForHTML specifies that '<', '>', and '&' characters within JSON strings
should be escaped as a hexadecimal Unicode codepoint (e.g., \u003c) so that
the output is safe to embed within HTML.
This only affects encoding and is ignored when decoding.
EscapeForJS specifies that U+2028 and U+2029 characters within JSON strings
should be escaped as a hexadecimal Unicode codepoint (e.g., \u2028) so that
the output is valid to embed within JavaScript. See RFC 8259, section 12.
This only affects encoding and is ignored when decoding.
Float constructs a Token representing a JSON number as
a 64-bit floating-point number formatted according to
ECMA-262, 10th edition, section 7.1.12.1 and RFC 8785, section 3.2.2.3.
with the exception that -0 is still formatted as -0.
The values NaN, +Inf, and -Inf will be represented
as a JSON string with the values "NaN", "Infinity", and "-Infinity".
Float32 constructs a Token representing a JSON number as
a 32-bit floating-point number formatted according to
ECMA-262, 10th edition, section 7.1.12.1,
with the exception that -0 is still formatted as -0.
The values NaN, +Inf, and -Inf will be represented
as a JSON string with the values "NaN", "Infinity", and "-Infinity".
Note that most JSON libraries and standards assume that JSON numbers
are 64-bit floating-point numbers. Use of 32-bit precision should
only be used if the corresponding decoder knows that
this JSON number token is expected to only have 32-bit precision.
For all other situations, prefer using the [Float] constructor instead.
Int constructs a Token representing a JSON number from an int64.
Multiline specifies that the JSON output should expand to multiple lines,
where every JSON object member or JSON array element appears on
a new, indented line according to the nesting depth.
If [SpaceAfterColon] is not specified, then the default is true.
If [SpaceAfterComma] is not specified, then the default is false.
If [WithIndent] is not specified, then the default is "\t".
If set to false, then the output is a single line,
where the only whitespace emitted is determined by the current
values of [SpaceAfterColon] and [SpaceAfterComma].
This only affects encoding and is ignored when decoding.
NewDecoder constructs a new streaming decoder reading from r.
If r is a [bytes.Buffer], then the decoder parses directly from the buffer
without first copying the contents to an intermediate buffer.
Additional writes to the buffer must not occur while the decoder is in use.
NewEncoder constructs a new streaming encoder writing to w
configured with the provided options.
It flushes the internal buffer when the buffer is sufficiently full or
when a top-level value has been written.
If w is a [bytes.Buffer], then the encoder appends directly into the buffer
without copying the contents from an intermediate buffer.
PreserveRawStrings specifies that when encoding a raw JSON string in a
[Token] or [Value], pre-escaped sequences
in a JSON string are preserved to the output.
However, raw strings still respect [EscapeForHTML] and [EscapeForJS]
such that the relevant characters are escaped.
If [AllowInvalidUTF8] is enabled, bytes of invalid UTF-8
are preserved to the output.
This only affects encoding and is ignored when decoding.
ReorderRawObjects specifies that when encoding a raw JSON object in a
[Value], the object members are reordered according to
RFC 8785, section 3.2.3.
This only affects encoding and is ignored when decoding.
SpaceAfterColon specifies that the JSON output should emit a space character
after each colon separator following a JSON object name.
If false, then no space character appears after the colon separator.
This only affects encoding and is ignored when decoding.
SpaceAfterComma specifies that the JSON output should emit a space character
after each comma separator following a JSON object value or array element.
If false, then no space character appears after the comma separator.
This only affects encoding and is ignored when decoding.
String constructs a Token representing a JSON string.
The provided string should contain valid UTF-8, otherwise invalid characters
may be mangled as the Unicode replacement character.
Uint constructs a Token representing a JSON number from a uint64.
WithIndent specifies that the encoder should emit multiline output
where each element in a JSON object or array begins on a new, indented line
beginning with the indent prefix (see [WithIndentPrefix])
followed by one or more copies of indent according to the nesting depth.
The indent must be composed of only space and tab characters.
If the intent is to emit indented output without a preference for
the particular indent string, then use [Multiline] instead.
This only affects encoding and is ignored when decoding.
Use of this option implies [Multiline] being set to true.
WithIndentPrefix specifies that the encoder should emit multiline output
where each element in a JSON object or array begins on a new, indented line
beginning with the indent prefix followed by one or more copies of indent
(see [WithIndent]) according to the nesting depth.
The prefix must be composed of only space and tab characters.
This only affects encoding and is ignored when decoding.
Use of this option implies [Multiline] being set to true.
mustReorderObjects reorders in-place all object members in a JSON value,
which must be valid otherwise it panics.
mustReorderObjectsFromDecoder recursively reorders all object members in place
according to the ordering specified in RFC 8785, section 3.2.3.
Pre-conditions:
- The value is valid (i.e., no decoder errors should ever occur).
- Initial call is provided a Decoder reading from the start of v.
Post-conditions:
- Exactly one JSON value is read from the Decoder.
- All fully-parsed JSON objects are reordered by directly moving
the members in the value buffer.
The runtime is approximately O(n·log(n)) + O(m·log(m)),
where n is len(v) and m is the total number of object members.
wrapSyntacticError wraps an error and annotates it with a precise location
using the provided [encoderState] or [decoderState].
If err is an [ioError] or [io.EOF], then it is not wrapped.
It takes a relative offset pos that can be resolved into
an absolute offset using state.offsetAt.
It takes a where that specifies how the JSON pointer is derived.
If the underlying error is a [pointerSuffixError],
then the suffix is appended to the derived pointer.
wrapWithArrayIndex wraps err with a JSON array index access.
wrapWithObjectName wraps err with a JSON object name access,
which must be a valid quoted JSON string.
Package-Level Variables (total 29, in which 10 are exported)
ErrDuplicateName indicates that a JSON token could not be
encoded or decoded because it results in a duplicate JSON object name.
This error is directly wrapped within a [SyntacticError] when produced.
The name of a duplicate JSON object member can be extracted as:
err := ...
serr, ok := errors.AsType[*jsontext.SyntacticError](err)
if ok && serr.Err == jsontext.ErrDuplicateName {
ptr := serr.JSONPointer // JSON pointer to duplicate name
name := ptr.LastToken() // duplicate name itself
...
}
This error is only returned if [AllowDuplicateNames] is false.
ErrNonStringName indicates that a JSON token could not be
encoded or decoded because it is not a string,
as required for JSON object names according to RFC 8259, section 4.
This error is directly wrapped within a [SyntacticError] when produced.
This does not own the internal buffer since it is externally provided.
This owns the internal buffer since there is no io.Writer to output to.
Since the buffer can get arbitrarily large in normal usage,
there is statistical tracking logic to determine whether to recycle
the internal buffer or not based on a history of utilization.
This does not own the internal buffer since
it is taken directly from the provided bytes.Buffer.
This does not own the internal buffer since
it is taken directly from the provided bytes.Buffer.
This owns the internal buffer, but it is only used to temporarily store
buffered JSON fetched from the underlying io.Reader.
In a sufficiently efficient streaming mode, we do not expect the buffer
to grow arbitrarily large. Thus, we avoid recycling large buffers.
This owns the internal buffer, but it is only used to temporarily store
buffered JSON before flushing it to the underlying io.Writer.
In a sufficiently efficient streaming mode, we do not expect the buffer
to grow arbitrarily large. Thus, we avoid recycling large buffers.