Involved Source Filesbuffer_pool.gobuffer_slice.go Package mem provides utilities that facilitate memory reuse in byte slices
that are used as buffers.
# Experimental
Notice: All APIs in this package are EXPERIMENTAL and may be changed or
removed in a later release.
Package-Level Type Names (total 13, in which 6 are exported)
/* sort exporteds by: | */
A Buffer represents a reference counted piece of data (in bytes) that can be
acquired by a call to NewBuffer() or Copy(). A reference to a Buffer may be
released by calling Free(), which invokes the free function given at creation
only after all references are released.
Note that a Buffer is not safe for concurrent access and instead each
goroutine should use its own reference to the data, which can be acquired via
a call to Ref().
Attempts to access the underlying data after releasing the reference to the
Buffer will panic. Free decrements this Buffer's reference counter and frees the underlying
byte slice if the counter reaches 0 as a result of this call. Len returns the Buffer's size. ReadOnlyData returns the underlying byte slice. Note that it is undefined
behavior to modify the contents of this slice in any way. Ref increases the reference counter for this Buffer.( Buffer) read(buf []byte) (int, Buffer)( Buffer) split(n int) (left, right Buffer)SliceBuffer
*bufferemptyBuffer
func Copy(data []byte, pool BufferPool) Buffer
func NewBuffer(data *[]byte, pool BufferPool) Buffer
func ReadUnsafe(dst []byte, buf Buffer) (int, Buffer)
func SplitUnsafe(buf Buffer, n int) (left, right Buffer)
func BufferSlice.MaterializeToBuffer(pool BufferPool) Buffer
func SliceBuffer.read(buf []byte) (int, Buffer)
func SliceBuffer.split(n int) (left, right Buffer)
func ReadUnsafe(dst []byte, buf Buffer) (int, Buffer)
func SplitUnsafe(buf Buffer, n int) (left, right Buffer)
BufferSlice offers a means to represent data that spans one or more Buffer
instances. A BufferSlice is meant to be immutable after creation, and methods
like Ref create and return copies of the slice. This is why all methods have
value receivers rather than pointer receivers.
Note that any of the methods that read the underlying buffers such as Ref,
Len or CopyTo etc., will panic if any underlying buffers have already been
freed. It is recommended to not directly interact with any of the underlying
buffers directly, rather such interactions should be mediated through the
various methods on this type.
By convention, any APIs that return (mem.BufferSlice, error) should reduce
the burden on the caller by never returning a mem.BufferSlice that needs to
be freed if the error is non-nil, unless explicitly stated. CopyTo copies each of the underlying Buffer's data into the given buffer,
returning the number of bytes copied. Has the same semantics as the copy
builtin in that it will copy as many bytes as it can, stopping when either dst
is full or s runs out of data, returning the minimum of s.Len() and len(dst). Free invokes Buffer.Free() on each Buffer in the slice. Len returns the sum of the length of all the Buffers in this slice.
# Warning
Invoking the built-in len on a BufferSlice will return the number of buffers
in the slice, and *not* the value returned by this function. Materialize concatenates all the underlying Buffer's data into a single
contiguous buffer using CopyTo. MaterializeToBuffer functions like Materialize except that it writes the data
to a single Buffer pulled from the given BufferPool.
As a special case, if the input BufferSlice only actually has one Buffer, this
function simply increases the refcount before returning said Buffer. Freeing this
buffer won't release it until the BufferSlice is itself released. Reader returns a new Reader for the input slice after taking references to
each underlying buffer. Ref invokes Ref on each buffer in the slice.
func ReadAll(r io.Reader, pool BufferPool) (BufferSlice, error)
func google.golang.org/grpc/encoding.CodecV2.Marshal(v any) (out BufferSlice, err error)
func google.golang.org/grpc/internal/transport.(*ClientStream).Read(n int) (BufferSlice, error)
func google.golang.org/grpc/internal/transport.(*ServerStream).Read(n int) (BufferSlice, error)
func google.golang.org/grpc.compress(in BufferSlice, cp grpc.Compressor, compressor encoding.Compressor, pool BufferPool) (BufferSlice, grpc.payloadFormat, error)
func google.golang.org/grpc.decompress(compressor encoding.Compressor, d BufferSlice, dc grpc.Decompressor, maxReceiveMessageSize int, pool BufferPool) (BufferSlice, error)
func google.golang.org/grpc.encode(c grpc.baseCodec, msg any) (BufferSlice, error)
func google.golang.org/grpc.msgHeader(data, compData BufferSlice, pf grpc.payloadFormat) (hdr []byte, payload BufferSlice)
func google.golang.org/grpc.prepareMsg(m any, codec grpc.baseCodec, cp grpc.Compressor, comp encoding.Compressor, pool BufferPool) (hdr []byte, data, payload BufferSlice, pf grpc.payloadFormat, err error)
func google.golang.org/grpc.recvAndDecompress(p *grpc.parser, s grpc.recvCompressor, dc grpc.Decompressor, maxReceiveMessageSize int, payInfo *grpc.payloadInfo, compressor encoding.Compressor, isServer bool) (out BufferSlice, err error)
func google.golang.org/grpc/internal/transport.(*Stream).read(n int) (data BufferSlice, err error)
func NewWriter(buffers *BufferSlice, pool BufferPool) io.Writer
func Reader.Reset(s BufferSlice)
func google.golang.org/grpc/encoding.CodecV2.Unmarshal(data BufferSlice, v any) error
func google.golang.org/grpc/internal/transport.(*ClientStream).Write(hdr []byte, data BufferSlice, opts *transport.WriteOptions) error
func google.golang.org/grpc/internal/transport.(*ServerStream).Write(hdr []byte, data BufferSlice, opts *transport.WriteOptions) error
func google.golang.org/grpc.compress(in BufferSlice, cp grpc.Compressor, compressor encoding.Compressor, pool BufferPool) (BufferSlice, grpc.payloadFormat, error)
func google.golang.org/grpc.decompress(compressor encoding.Compressor, d BufferSlice, dc grpc.Decompressor, maxReceiveMessageSize int, pool BufferPool) (BufferSlice, error)
func google.golang.org/grpc.msgHeader(data, compData BufferSlice, pf grpc.payloadFormat) (hdr []byte, payload BufferSlice)
NopBufferPool is a buffer pool that returns new buffers without pooling. Get returns a buffer with specified length from the pool. Put returns a buffer to the pool.
NopBufferPool : BufferPool
Reader exposes a BufferSlice's data as an io.Reader, allowing it to interface
with other parts systems. It also provides an additional convenience method
Remaining(), which returns the number of unread bytes remaining in the slice.
Buffers will be freed as they are read. Close frees the underlying BufferSlice and never returns an error. Subsequent
calls to Read will return (0, io.EOF).( Reader) Read(p []byte) (n int, err error)( Reader) ReadByte() (byte, error) Remaining returns the number of unread bytes remaining in the slice. Reset frees the currently held buffer slice and starts reading from the
provided slice. This allows reusing the reader object.
*sliceReader
Reader : compress/flate.Reader
Reader : io.ByteReader
Reader : io.Closer
Reader : io.ReadCloser
Reader : io.Reader
func BufferSlice.Reader() Reader
SliceBuffer is a Buffer implementation that wraps a byte slice. It provides
methods for reading, splitting, and managing the byte slice. Free is a noop implementation of Free. Len is a noop implementation of Len. ReadOnlyData returns the byte slice. Ref is a noop implementation of Ref.( SliceBuffer) read(buf []byte) (int, Buffer)( SliceBuffer) split(n int) (left, right Buffer)
SliceBuffer : Buffer
simpleBufferPool is an implementation of the BufferPool interface that
attempts to pool buffers with a sync.Pool. When Get is invoked, it tries to
acquire a buffer from the pool but if that buffer is too small, it returns it
to the pool and creates a new one.poolsync.Pool(*simpleBufferPool) Get(size int) *[]byte(*simpleBufferPool) Put(buf *[]byte)
*simpleBufferPool : BufferPool
sizedBufferPool is a BufferPool implementation that is optimized for specific
buffer sizes. For example, HTTP/2 frames within gRPC have a default max size
of 16kb and a sizedBufferPool can be configured to only return buffers with a
capacity of 16kb. Note that however it does not support returning larger
buffers and in fact panics if such a buffer is requested. Because of this,
this BufferPool implementation is not meant to be used on its own and rather
is intended to be embedded in a tieredBufferPool such that Get is only
invoked when the required size is smaller than or equal to defaultSize.defaultSizeintpoolsync.Pool(*sizedBufferPool) Get(size int) *[]byte(*sizedBufferPool) Put(buf *[]byte)
*sizedBufferPool : BufferPool
func newSizedBufferPool(size int) *sizedBufferPool
Package-Level Functions (total 12, in which 9 are exported)
Copy creates a new Buffer from the given data, initializing the reference
counter to 1.
It acquires a []byte from the given pool and copies over the backing array
of the given data. The []byte acquired from the pool is returned to the
pool when all references to the returned Buffer are released.
DefaultBufferPool returns the current default buffer pool. It is a BufferPool
created with NewBufferPool that uses a set of default sizes optimized for
expected workflows.
IsBelowBufferPoolingThreshold returns true if the given size is less than or
equal to the threshold for buffer pooling. This is used to determine whether
to pool buffers or allocate them directly.
NewBuffer creates a new Buffer from the given data, initializing the reference
counter to 1. The data will then be returned to the given pool when all
references to the returned Buffer are released. As a special case to avoid
additional allocations, if the given buffer pool is nil, the returned buffer
will be a "no-op" Buffer where invoking Buffer.Free() does nothing and the
underlying data is never freed.
Note that the backing array of the given data is not copied.
NewTieredBufferPool returns a BufferPool implementation that uses multiple
underlying pools of the given pool sizes.
NewWriter wraps the given BufferSlice and BufferPool to implement the
io.Writer interface. Every call to Write copies the contents of the given
buffer into a new Buffer pulled from the given pool and the Buffer is
added to the given BufferSlice.
ReadAll reads from r until an error or EOF and returns the data it read.
A successful call returns err == nil, not err == EOF. Because ReadAll is
defined to read from src until EOF, it does not treat an EOF from Read
as an error to be reported.
Important: A failed call returns a non-nil error and may also return
partially read buffers. It is the responsibility of the caller to free the
BufferSlice returned, or its memory will not be reused.
ReadUnsafe reads bytes from the given Buffer into the provided slice.
It does not perform safety checks.
SplitUnsafe modifies the receiver to point to the first n bytes while it
returns a new reference to the remaining bytes. The returned Buffer
functions just like a normal reference acquired using Ref().