Involved Source Filesbuffer_pool.gobuffer_slice.go Package mem provides utilities that facilitate memory reuse in byte slices
that are used as buffers.
# Experimental
Notice: All APIs in this package are EXPERIMENTAL and may be changed or
removed in a later release.
Package-Level Type Names (total 9, in which 6 are exported)
/* sort exporteds by: | */
A Buffer represents a reference counted piece of data (in bytes) that can be
acquired by a call to NewBuffer() or Copy(). A reference to a Buffer may be
released by calling Free(), which invokes the free function given at creation
only after all references are released.
Note that a Buffer is not safe for concurrent access and instead each
goroutine should use its own reference to the data, which can be acquired via
a call to Ref().
Attempts to access the underlying data after releasing the reference to the
Buffer will panic. Free decrements this Buffer's reference counter and frees the underlying
byte slice if the counter reaches 0 as a result of this call. Len returns the Buffer's size. ReadOnlyData returns the underlying byte slice. Note that it is undefined
behavior to modify the contents of this slice in any way. Ref increases the reference counter for this Buffer. Slice returns a new Buffer that is a view into this buffer's data
from [start:end). The buffer is not modified. Panics if the buffer
has been freed or if start/end are out of bounds.( Buffer) read(buf []byte) (int, Buffer)( Buffer) split(n int) (left, right Buffer)SliceBuffer
*bufferemptyBuffer
func Copy(data []byte, pool BufferPool) Buffer
func NewBuffer(data *[]byte, pool BufferPool) Buffer
func ReadUnsafe(dst []byte, buf Buffer) (int, Buffer)
func SplitUnsafe(buf Buffer, n int) (left, right Buffer)
func Buffer.Slice(start, end int) Buffer
func BufferSlice.MaterializeToBuffer(pool BufferPool) Buffer
func SliceBuffer.Slice(start, end int) Buffer
func SliceBuffer.read(buf []byte) (int, Buffer)
func SliceBuffer.split(n int) (left, right Buffer)
func ReadUnsafe(dst []byte, buf Buffer) (int, Buffer)
func SplitUnsafe(buf Buffer, n int) (left, right Buffer)
BufferSlice offers a means to represent data that spans one or more Buffer
instances. A BufferSlice is meant to be immutable after creation, and methods
like Ref create and return copies of the slice. This is why all methods have
value receivers rather than pointer receivers.
Note that any of the methods that read the underlying buffers such as Ref,
Len or CopyTo etc., will panic if any underlying buffers have already been
freed. It is recommended to not directly interact with any of the underlying
buffers directly, rather such interactions should be mediated through the
various methods on this type.
By convention, any APIs that return (mem.BufferSlice, error) should reduce
the burden on the caller by never returning a mem.BufferSlice that needs to
be freed if the error is non-nil, unless explicitly stated. CopyTo copies each of the underlying Buffer's data into the given buffer,
returning the number of bytes copied. Has the same semantics as the copy
builtin in that it will copy as many bytes as it can, stopping when either dst
is full or s runs out of data, returning the minimum of s.Len() and len(dst). Free invokes Buffer.Free() on each Buffer in the slice. Len returns the sum of the length of all the Buffers in this slice.
# Warning
Invoking the built-in len on a BufferSlice will return the number of buffers
in the slice, and *not* the value returned by this function. Materialize concatenates all the underlying Buffer's data into a single
contiguous buffer using CopyTo. MaterializeToBuffer functions like Materialize except that it writes the data
to a single Buffer pulled from the given BufferPool.
As a special case, if the input BufferSlice only actually has one Buffer, this
function simply increases the refcount before returning said Buffer. Freeing this
buffer won't release it until the BufferSlice is itself released. Reader returns a new Reader for the input slice after taking references to
each underlying buffer. Ref invokes Ref on each buffer in the slice.
func ReadAll(r io.Reader, pool BufferPool) (BufferSlice, error)
func google.golang.org/grpc/encoding.CodecV2.Marshal(v any) (out BufferSlice, err error)
func google.golang.org/grpc/internal/transport.(*ClientStream).Read(n int) (BufferSlice, error)
func google.golang.org/grpc/internal/transport.(*ServerStream).Read(n int) (BufferSlice, error)
func google.golang.org/grpc.compress(in BufferSlice, cp grpc.Compressor, compressor encoding.Compressor, pool BufferPool) (BufferSlice, grpc.payloadFormat, error)
func google.golang.org/grpc.decompress(compressor encoding.Compressor, d BufferSlice, dc grpc.Decompressor, maxReceiveMessageSize int, pool BufferPool) (BufferSlice, error)
func google.golang.org/grpc.encode(c grpc.baseCodec, msg any) (BufferSlice, error)
func google.golang.org/grpc.msgHeader(data, compData BufferSlice, pf grpc.payloadFormat) (hdr []byte, payload BufferSlice)
func google.golang.org/grpc.prepareMsg(m any, codec grpc.baseCodec, cp grpc.Compressor, comp encoding.Compressor, pool BufferPool) (hdr []byte, data, payload BufferSlice, pf grpc.payloadFormat, err error)
func google.golang.org/grpc.recvAndDecompress(p *grpc.parser, s grpc.recvCompressor, dc grpc.Decompressor, maxReceiveMessageSize int, payInfo *grpc.payloadInfo, compressor encoding.Compressor, isServer bool) (out BufferSlice, err error)
func google.golang.org/grpc/internal/transport.(*Stream).read(n int) (data BufferSlice, err error)
func NewWriter(buffers *BufferSlice, pool BufferPool) io.Writer
func (*Reader).Reset(s BufferSlice)
func google.golang.org/grpc/encoding.CodecV2.Unmarshal(data BufferSlice, v any) error
func google.golang.org/grpc/internal/transport.(*ClientStream).Write(hdr []byte, data BufferSlice, opts *transport.WriteOptions) error
func google.golang.org/grpc/internal/transport.(*ServerStream).Write(hdr []byte, data BufferSlice, opts *transport.WriteOptions) error
func google.golang.org/grpc.compress(in BufferSlice, cp grpc.Compressor, compressor encoding.Compressor, pool BufferPool) (BufferSlice, grpc.payloadFormat, error)
func google.golang.org/grpc.decompress(compressor encoding.Compressor, d BufferSlice, dc grpc.Decompressor, maxReceiveMessageSize int, pool BufferPool) (BufferSlice, error)
func google.golang.org/grpc.msgHeader(data, compData BufferSlice, pf grpc.payloadFormat) (hdr []byte, payload BufferSlice)
NopBufferPool is a buffer pool that returns new buffers without pooling.NopBufferPoolmem.NopBufferPool Get returns a buffer with specified length from the pool. Put returns a buffer to the pool.
NopBufferPool : BufferPool
NopBufferPool : google.golang.org/grpc/internal/mem.bufferPool
Reader exposes a BufferSlice's data as an io.Reader, allowing it to interface
with other systems.
Buffers will be freed as they are read.
A Reader can be constructed from a BufferSlice; alternatively the zero value
of a Reader may be used after calling Reset on it. The index into data[0].ReadOnlyData().dataBufferSlicelenint Close frees the underlying BufferSlice and never returns an error. Subsequent
calls to Read will return (0, io.EOF). Discard skips the next n bytes, returning the number of bytes discarded.
It frees buffers as they are fully consumed.
If Discard skips fewer than n bytes, it also returns an error. Peek returns the next n bytes without advancing the reader.
Peek appends results to the provided res slice and returns the updated slice.
This pattern allows re-using the storage of res if it has sufficient
capacity.
The returned subslices are views into the underlying buffers and are only
valid until the reader is advanced past the corresponding buffer.
If Peek returns fewer than n bytes, it also returns an error.(*Reader) Read(buf []byte) (n int, _ error) ReadByte reads a single byte. Remaining returns the number of unread bytes remaining in the slice. Reset frees the currently held buffer slice and starts reading from the
provided slice. This allows reusing the reader object.(*Reader) freeFirstBufferIfEmpty() bool
*Reader : compress/flate.Reader
*Reader : io.ByteReader
*Reader : io.Closer
*Reader : io.ReadCloser
*Reader : io.Reader
func BufferSlice.Reader() *Reader
SliceBuffer is a Buffer implementation that wraps a byte slice. It provides
methods for reading, splitting, and managing the byte slice. Free is a noop implementation of Free. Len is a noop implementation of Len. ReadOnlyData returns the byte slice. Ref is a noop implementation of Ref. Slice returns a new SliceBuffer that is a view into the receiver from [start:end).( SliceBuffer) read(buf []byte) (int, Buffer)( SliceBuffer) split(n int) (left, right Buffer)
SliceBuffer : Buffer
data[]byte The following fields are only set for root buffers.poolBufferPoolrefsatomic.Int32 rootBuf is the buffer responsible for returning origData to the pool
once the reference count drops to 0.
When a buffer is split, the new buffer inherits the rootBuf of the
original and increments the root's reference count. For the
initial buffer (the root), this field points to itself.(*buffer) Free()(*buffer) Len() int(*buffer) ReadOnlyData() []byte(*buffer) Ref()(*buffer) Slice(start, end int) Buffer(*buffer) String() string(*buffer) read(buf []byte) (int, Buffer)(*buffer) split(n int) (Buffer, Buffer)
*buffer : Buffer
*buffer : expvar.Var
*buffer : fmt.Stringer
*buffer : context.stringer
*buffer : runtime.stringer
func newBuffer() *buffer
Package-Level Functions (total 12, in which 10 are exported)
Copy creates a new Buffer from the given data, initializing the reference
counter to 1.
It acquires a []byte from the given pool and copies over the backing array
of the given data. The []byte acquired from the pool is returned to the
pool when all references to the returned Buffer are released.
DefaultBufferPool returns the current default buffer pool. It is a BufferPool
created with NewBufferPool that uses a set of default sizes optimized for
expected workflows.
IsBelowBufferPoolingThreshold returns true if the given size is less than or
equal to the threshold for buffer pooling. This is used to determine whether
to pool buffers or allocate them directly.
NewBinaryTieredBufferPool returns a BufferPool backed by multiple sub-pools.
This structure enables O(1) lookup time for Get and Put operations.
The arguments provided are the exponents for the buffer capacities (powers
of 2), not the raw byte sizes. For example, to create a pool of 16KB buffers
(2^14 bytes), pass 14 as the argument.
NewBuffer creates a new Buffer from the given data, initializing the reference
counter to 1. The data will then be returned to the given pool when all
references to the returned Buffer are released. As a special case to avoid
additional allocations, if the given buffer pool is nil, the returned buffer
will be a "no-op" Buffer where invoking Buffer.Free() does nothing and the
underlying data is never freed.
Note that the backing array of the given data is not copied.
NewTieredBufferPool returns a BufferPool implementation that uses multiple
underlying pools of the given pool sizes.
NewWriter wraps the given BufferSlice and BufferPool to implement the
io.Writer interface. Every call to Write copies the contents of the given
buffer into a new Buffer pulled from the given pool and the Buffer is
added to the given BufferSlice.
ReadAll reads from r until an error or EOF and returns the data it read.
A successful call returns err == nil, not err == EOF. Because ReadAll is
defined to read from src until EOF, it does not treat an EOF from Read
as an error to be reported.
Important: A failed call returns a non-nil error and may also return
partially read buffers. It is the responsibility of the caller to free the
BufferSlice returned, or its memory will not be reused.
ReadUnsafe reads bytes from the given Buffer into the provided slice.
It does not perform safety checks.
SplitUnsafe modifies the receiver to point to the first n bytes while it
returns a new reference to the remaining bytes. The returned Buffer
functions just like a normal reference acquired using Ref().