Loading...
Searching...
No Matches
StreamCompression.h File Reference

Convenience wrappers to using Blosc and reading and writing of Paged data. More...

`#include <openvdb/io/io.h>`
`#include <tbb/spin_mutex.h>`
`#include <memory>`
`#include <string>`

Go to the source code of this file.

## Classes

class | Page
| Stores a variable-size, compressed, delayed-load Page of data that is loaded into memory when accessed. Access to the Page is thread-safe as loading and decompressing the data is protected by a mutex. More...
class | PageHandle
| A PageHandle holds a unique ptr to a Page and a specific stream pointer to a point within the decompressed Page buffer. More...
class | PagedInputStream
| A Paging wrapper to std::istream that is responsible for reading from a given input stream and creating Page objects and PageHandles that reference those pages for delayed reading. More...
class | PagedOutputStream
| A Paging wrapper to std::ostream that is responsible for writing from a given output stream at intervals set by the PageSize. As Pages are variable in size, they are flushed to disk as soon as sufficiently large. More...

## Namespaces

namespace | openvdb
namespace | openvdb::v11_0
namespace | openvdb::v11_0::compression

## Functions

OPENVDB_API bool | bloscCanCompress ()
| Returns true if compression is available.
OPENVDB_API size_t | bloscUncompressedSize (const char *buffer)
| Retrieves the uncompressed size of buffer when uncompressed.
OPENVDB_API void | bloscCompress (char *compressedBuffer, size_t &compressedBytes, const size_t bufferBytes, const char *uncompressedBuffer, const size_t uncompressedBytes)
| Compress into the supplied buffer.
OPENVDB_API std::unique_ptr< char[]> | bloscCompress (const char *buffer, const size_t uncompressedBytes, size_t &compressedBytes, const bool resize=true)
| Compress and return the heap-allocated compressed buffer.
OPENVDB_API size_t | bloscCompressedSize (const char *buffer, const size_t uncompressedBytes)
| Convenience wrapper to retrieve the compressed size of buffer when compressed.
OPENVDB_API void | bloscDecompress (char *uncompressedBuffer, const size_t expectedBytes, const size_t bufferBytes, const char *compressedBuffer)
| Decompress into the supplied buffer. Will throw if decompression fails or uncompressed buffer has insufficient space in which to decompress.
OPENVDB_API std::unique_ptr< char[]> | bloscDecompress (const char *buffer, const size_t expectedBytes, const bool resize=true)
| Decompress and return the the heap-allocated uncompressed buffer.

## Variables

static const int | BLOSC_MINIMUM_BYTES = 48
static const int | BLOSC_PAD_BYTES = 128
static const int | PageSize = 1024 * 1024

## Detailed Description

Convenience wrappers to using Blosc and reading and writing of Paged data.

Author
  Dan Bailey

Blosc is most effective with large (> ~256KB) blocks of data. Writing the entire data block contiguously would provide the most optimal compression, however would limit the ability to use delayed-loading as the whole block would be required to be loaded from disk at once. To balance these two competing factors, Paging is used to write out blocks of data that are a reasonable size for Blosc. These Pages are loaded lazily, tracking the input stream pointers and creating Handles that reference portions of the buffer. When the Page buffer is accessed, the data will be read from the stream.