Scarfski/bitsery
1
0
Fork 0
mirror of https://github.com/fraillt/bitsery.git synced 2026-08-08 14:29:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactoring to work with C++11, and added containerMap extension

Browse Source
This commit is contained in:
fraillt
2017-09-19 11:13:59 +03:00
parent 2f8ae0075c
commit ad7090539e
25 changed files with 713 additions and 251 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
CHANGELOG.md
View File

@@ -2,19 +2,23 @@
### Features
* now works with C++11 compiler.
* new function **growable**, that allows to have forward/backward compatability within this functions serialization flow. It only allows to append new data at the end of to existing flow without breaking old consumers.
* old consumer: correctly read old interfce and ignore new data.
* new consumer: get defaults (zero values) for new fields, when reading old data.
* new **SERIALIZE_FRIEND** macro that enables to serialize private object fields.
* added new extension for associative *map* containers **containerMap**.
* friendly static_assert message when serializing **object**, that doesn't have **serialize** function defined.
* added **object** overload, that invokes user function/lambda with object. It is the same as calling user function directly, but makes more consistent API.
* Serializer/Deserializer now have optional *context* parameter, that might be required in some specific serialization cases.
* improved serialization performance: added support for fixed size buffer for best performance.
* added traits for custom types specialization, in *details/traits.h*
* improved serialization performance: added support for fixed size buffer for BufferWriter for best performance.
* improved performance for reading/writing container sizes.
* added new method to BufferReader **getError** which returns on of three values: NO_ERROR, BUFFER_OVERFLOW, INVALID_BUFFER_DATA, also added setError method, that is only used by Deserializer.
### Breaking changes
* removed **SERIALIZE** macro, and changed interface for all functions that use custom lambdas, to work with C++11. Now lambda function must capture serializer/deserializer.
* container and text sizes representation changed, to allow much faster size reads/writes for small values.
* renamed functions:
* **ext** to **extend** and changed its interface, to make it more easy to extend.
@@ -23,7 +27,7 @@
* now all serializer/deserializer functions return void, to avoid undefined behaviour for functions parameters evaluation when using method chaining. There was no benefits apart from *nicer* syntax, but could have undefined behaviour when building complex serialization flows.
* removed **array** and added fixed sizes overloads for **container**.
* changed BufferWriter/Reader behaviour:
* added *FixedBufferSize* config bool parameter for *BufferWriter* for better serializer performance (more than 50% improvement). Default config is resizable buffer (*std::vector<uint8_t>*).
* added support for fixed size buffers for better serializer performance (more than 50% improvement). Default config is resizable buffer (*std::vector<uint8_t>*).
* after serialization, call *getWrittenRange* to get valid range written to buffer, because BufferWritter for resizable buffer now always resize to *capacity* to avoid using *back_insert_iterator* for better performance.
* BufferReader has constructor with iterators (range), and raw value type pointers (begin, end).
* removed **isValid** method from Deserializer, only BufferReader/Writer store states.
@@ -32,8 +36,7 @@
### Bug fixes
* **readBytes** was reading in aligned mode, when scratch value from previous bit operations was zero.
* **writeBits** had incorrect assert when writing negative int64_t.
<a name="2.0.1"></a>
# [2.0.1](https://github.com/fraillt/bitsery/compare/v2.0.0...v2.0.1) (2017-08-12)

24
README.md
View File

@@ -14,15 +14,29 @@ All cross-platform requirements are enforced at compile time, so serialized data
* Cross-platform compatible.
* Optimized for speed and space.
* No code generation required: no IDL or metadata, use your types directly.
* 2-in-1 declarative control flow, same code for serialization and deserialization.
* No code generation required: no IDL or metadata, just use your types directly.
* Runtime error checking on deserialization,- designed to be save with untrusted network data.
* Provides forward/backward compatibility for your types.
* 2-in-1 declarative control flow, same code for serialization and deserialization.
* Allows fine-grained serialization control using advanced serialization techniques like value ranges, entrophy encoding etc...
* Extendable for types that requires different serialization and deserialization logic (e.g. pointers, or geometry compression).
* Extendable for flows that requires different serialization and deserialization logic (e.g. pointers serialization, geometry compression, or versioning support).
* Configurable endianess support.
* No macros.
## Why to use bitsery
Read more about the "why" in library [motivation](doc/design/README.md) section.
Look at the numbers, and features list, and deside for your self. *(benchmarked on Ubuntu with GCC 7.1)*
| | serialize | deserialize | data size | executable size |
|-----------------------------------------------------------|-----------|-------------|-------------|-----------------|
| flatbuffers | 1852 ms. | 777 ms. | 27252 bytes | 74544 bytes |
| cereal | 1069 ms. | 1385 ms. | 20208 bytes | 72336 bytes |
| bitsery | 808 ms. | 737 ms. | 14803 bytes | 69784 bytes |
| bitsery fixed-size buffer | 297 ms. | 738 ms. | 14803 bytes | 69928 bytes |
| bitsery optimized serialization flow | 686 ms. | 997 ms. | 6601 bytes | 69320 bytes |
| bitsery optimized serialization flow + fixed-size buffer | 446 ms. | 996 ms. | 6601 bytes | 69464 bytes |
If still not convinced read more in library [motivation](doc/design/README.md) section.
## How to use it
This documentation comprises these parts:
@@ -31,6 +45,8 @@ This documentation comprises these parts:
## Requirements
Works with C++11 compiler, no additional dependencies, include `<bitsery/bitsery.h>` and you're done.
## Platforms
This library was tested on

1
doc/design/README.md
View File

@@ -1,6 +1,7 @@
## Motivation
Inspiration to create **bitsery** came mainly because there aren't any good alternatives for C++.
I wanted serializer that is easy to use like [cereal](http://uscilab.github.io/cereal/)
Most well-known serialization libraries are *too fat* and tries to solve too many things by supporting multiple data formats (binary, json, xml) and multiple languages (C++, C#, Javascript, etc..) while in the process becomes hard to use, are memory or/and speed inefficient.
The best alternative that I was able to find is [flatbuffers](https://google.github.io/flatbuffers/).

7
examples/basic_usage.cpp
View File

@@ -8,7 +8,8 @@ struct MyStruct {
};
//define how object should be serialized/deserialized
SERIALIZE(MyStruct) {
template <typename S>
void serialize(S& s, MyStruct& o) {
s.value4b(o.i);
s.value2b(o.e);
s.container4b(o.fs, 10);
@@ -27,7 +28,7 @@ int main() {
//2) create buffer writer that is able to write bytes or bits to buffer
BufferWriter bw{buffer};
//3) create serializer
Serializer<BufferWriter> ser{bw};
Serializer ser{bw};
//serialize object, can also be invoked like this: serialize(ser, data)
ser.object(data);
@@ -39,7 +40,7 @@ int main() {
//1) create buffer reader
BufferReader br{bw.getWrittenRange()};
//2) create deserializer
Deserializer<BufferReader> des{br};
Deserializer des{br};
//deserialize same object, can also be invoked like this: serialize(des, data)
des.object(res);

14
include/bitsery/buffer_reader.h
View File

@@ -35,8 +35,8 @@ namespace bitsery {
struct BasicBufferReader {
using BufferType = typename Config::BufferType;
using ValueType = typename BufferType::value_type;
using BufferIteratorType = typename BufferType::iterator;
using ValueType = typename details::BufferContainerTraits<BufferType>::TValue;
using BufferIteratorType = typename details::BufferContainerTraits<BufferType>::TIterator;
using ScratchType = typename details::SCRATCH_TYPE<ValueType>::type;
BasicBufferReader(ValueType* begin, ValueType* end)
@@ -78,7 +78,7 @@ namespace bitsery {
if (!m_scratchBits)
directRead(&v, 1);
else
readBits(reinterpret_cast<UT &>(v), details::BITS_SIZE<T>);
readBits(reinterpret_cast<UT &>(v), details::BITS_SIZE<T>::value);
}
template<size_t SIZE, typename T>
@@ -93,7 +93,7 @@ namespace bitsery {
//todo improve implementation
const auto end = buf + count;
for (auto it = buf; it != end; ++it)
readBits(reinterpret_cast<UT &>(*it), details::BITS_SIZE<T>);
readBits(reinterpret_cast<UT &>(*it), details::BITS_SIZE<T>::value);
}
}
@@ -160,7 +160,7 @@ namespace bitsery {
static_assert(!std::is_const<T>::value, "");
const auto bytesCount = sizeof(T) * count;
if (std::distance(_pos, _end) >= static_cast<typename BufferType::difference_type>(bytesCount)) {
if (std::distance(_pos, _end) >= static_cast<typename details::BufferContainerTraits<BufferType>::TDifference>(bytesCount)) {
std::memcpy(reinterpret_cast<ValueType *>(v), _pos, bytesCount);
_pos += bytesCount;
@@ -192,12 +192,12 @@ namespace bitsery {
auto bitsLeft = size;
T res{};
while (bitsLeft > 0) {
auto bits = std::min(bitsLeft, details::BITS_SIZE<ValueType>);
auto bits = std::min(bitsLeft, details::BITS_SIZE<ValueType>::value);
if (m_scratchBits < bits) {
ValueType tmp;
directRead(&tmp, 1);
m_scratch |= static_cast<ScratchType>(tmp) << m_scratchBits;
m_scratchBits += details::BITS_SIZE<ValueType>;
m_scratchBits += details::BITS_SIZE<ValueType>::value;
}
auto shiftedRes =
static_cast<T>(m_scratch & ((static_cast<ScratchType>(1) << bits) - 1)) << (size - bitsLeft);

35
include/bitsery/buffer_writer.h
View File

@@ -37,13 +37,13 @@ namespace bitsery {
void writeBytes(const T &) {
static_assert(std::is_integral<T>(), "");
static_assert(sizeof(T) == SIZE, "");
_bitsCount += details::BITS_SIZE<T>;
_bitsCount += details::BITS_SIZE<T>::value;
}
template<typename T>
void writeBits(const T &, size_t bitsCount) {
static_assert(std::is_integral<T>() && std::is_unsigned<T>(), "");
assert(bitsCount <= details::BITS_SIZE<T>);
assert(bitsCount <= details::BITS_SIZE<T>::value);
_bitsCount += bitsCount;
}
@@ -51,7 +51,7 @@ namespace bitsery {
void writeBuffer(const T *, size_t count) {
static_assert(std::is_integral<T>(), "");
static_assert(sizeof(T) == SIZE, "");
_bitsCount += details::BITS_SIZE<T> * count;
_bitsCount += details::BITS_SIZE<T>::value * count;
}
void align() {
@@ -95,15 +95,14 @@ namespace bitsery {
template<typename Config>
struct BasicBufferWriter {
using BufferType = typename Config::BufferType;
using ValueType = typename BufferType::value_type;
using ValueType = typename details::BufferContainerTraits<BufferType>::TValue;
using ScratchType = typename details::SCRATCH_TYPE<ValueType>::type;
using BufferContext = details::WriteBufferContext<BufferType, Config::FixedBufferSize>;
using BufferContext = details::WriteBufferContext<BufferType, details::BufferContainerTraits<BufferType>::isResizable>;
explicit BasicBufferWriter(BufferType &buffer)
: _bufferContext{buffer}
{
static_assert(std::is_unsigned<ValueType>(), "Config::BufferType::value_type must be unsigned");
static_assert(std::is_unsigned<ScratchType>(), "Config::BufferScrathType must be unsigned");
static_assert(std::is_unsigned<ValueType>(), "Config::BufferType value type must be unsigned");
static_assert(sizeof(ValueType) * 2 == sizeof(ScratchType),
"ScratchType must be 2x bigger than value type");
static_assert(sizeof(ValueType) == 1, "currently only supported BufferValueType is 1 byte");
@@ -128,7 +127,7 @@ namespace bitsery {
directWrite(&v, 1);
} else {
using UT = typename std::make_unsigned<T>::type;
writeBits(reinterpret_cast<const UT &>(v), details::BITS_SIZE<T>);
writeBits(reinterpret_cast<const UT &>(v), details::BITS_SIZE<T>::value);
}
}
@@ -143,20 +142,22 @@ namespace bitsery {
//todo improve implementation
const auto end = buf + count;
for (auto it = buf; it != end; ++it)
writeBits(reinterpret_cast<const UT &>(*it), details::BITS_SIZE<T>);
writeBits(reinterpret_cast<const UT &>(*it), details::BITS_SIZE<T>::value);
}
}
template<typename T>
void writeBits(const T &v, size_t bitsCount) {
static_assert(std::is_integral<T>() && std::is_unsigned<T>(), "");
assert(0 < bitsCount && bitsCount <= details::BITS_SIZE<T>);
assert(v <= ((1ULL << bitsCount) - 1));
assert(0 < bitsCount && bitsCount <= details::BITS_SIZE<T>::value);
assert(v <= (bitsCount < 64
? (1ULL << bitsCount) - 1
: (1ULL << (bitsCount-1)) + ((1ULL << (bitsCount-1)) -1)));
writeBitsInternal(v, bitsCount);
}
void align() {
writeBitsInternal(ValueType{}, (details::BITS_SIZE<ValueType> - _scratchBits) % 8);
writeBitsInternal(ValueType{}, (details::BITS_SIZE<ValueType>::value - _scratchBits) % 8);
}
void flush() {
@@ -164,7 +165,7 @@ namespace bitsery {
_session.flushSessions(*this);
}
BufferRange<typename BufferType::iterator> getWrittenRange() const {
BufferRange<typename details::BufferContainerTraits<BufferType>::TIterator> getWrittenRange() const {
return _bufferContext.getWrittenRange();
}
@@ -202,7 +203,7 @@ namespace bitsery {
template<typename T>
void writeBitsInternal(const T &v, size_t size) {
constexpr size_t valueSize = details::BITS_SIZE<ValueType>;
constexpr size_t valueSize = details::BITS_SIZE<ValueType>::value;
auto value = v;
auto bitsLeft = size;
while (bitsLeft > 0) {
@@ -226,11 +227,11 @@ namespace bitsery {
if (size > 0) {
_scratch |= static_cast<ScratchType>( v ) << _scratchBits;
_scratchBits += size;
if (_scratchBits >= details::BITS_SIZE<ValueType>) {
if (_scratchBits >= details::BITS_SIZE<ValueType>::value) {
auto tmp = static_cast<ValueType>(_scratch & _MASK);
directWrite(&tmp, 1);
_scratch >>= details::BITS_SIZE<ValueType>;
_scratchBits -= details::BITS_SIZE<ValueType>;
_scratch >>= details::BITS_SIZE<ValueType>::value;
_scratchBits -= details::BITS_SIZE<ValueType>::value;
}
}
}

15
include/bitsery/common.h
View File

@@ -29,25 +29,12 @@
namespace bitsery {
//default configuration for buffer writing/reading operations
struct DefaultConfig {
static constexpr EndiannessType NetworkEndianness = EndiannessType::LittleEndian;
static constexpr bool FixedBufferSize = false;//false means that buffer is resizable and will be used back_insert_iterator for insertion, for reading has no effect.
using BufferType = std::vector<uint8_t>;//buffer value type must be unsigned, currently only uint8_t supported
};
/*
* serializer macro, serialize function specialization that accepts T& and const T&
*/
#define SERIALIZE(ObjectType) \
template <typename S, typename T, typename std::enable_if<std::is_same<T, ObjectType>::value || std::is_same<T, const ObjectType>::value>::type* = nullptr> \
void serialize(S& s, T& o)
#define SERIALIZE_FRIEND(ObjectType) \
template <typename S, typename T, typename std::enable_if<std::is_same<T, ObjectType>::value || std::is_same<T, const ObjectType>::value>::type* = nullptr> \
friend void serialize(S& s, T& o)
}
#endif //BITSERY_COMMON_H

102
include/bitsery/deserializer.h
View File

@@ -32,10 +32,10 @@
namespace bitsery {
template<typename Reader>
class Deserializer {
template<typename Config>
class BasicDeserializer {
public:
Deserializer(Reader &r, void* context = nullptr) : _reader{r}, _context{context} {};
explicit BasicDeserializer(BasicBufferReader<Config> &r, void* context = nullptr) : _reader{r}, _context{context} {};
/*
* get serialization context.
@@ -51,12 +51,12 @@ namespace bitsery {
template<typename T>
void object(T &&obj) {
details::SerializeFunction<Deserializer, T>::invoke(*this, std::forward<T>(obj));
details::SerializeFunction<BasicDeserializer, T>::invoke(*this, std::forward<T>(obj));
}
template<typename T, typename Fnc>
void object(T &&obj, Fnc &&fnc) {
fnc(*this, std::forward<T>(obj));
fnc(std::forward<T>(obj));
};
/*
@@ -71,7 +71,7 @@ namespace bitsery {
template<size_t VSIZE, typename T, typename std::enable_if<std::is_enum<T>::value>::type * = nullptr>
void value(T &v) {
using UT = std::underlying_type_t<T>;
using UT = typename std::underlying_type<T>::type;
_reader.template readBytes<VSIZE>(reinterpret_cast<UT &>(v));
}
@@ -87,7 +87,7 @@ namespace bitsery {
template <typename T, typename Fnc>
void growable(T&& obj, Fnc&& fnc) {
_reader.beginSession();
fnc(*this, std::forward<T>(obj));
fnc(std::forward<T>(obj));
_reader.endSession();
};
@@ -102,12 +102,16 @@ namespace bitsery {
template<size_t VSIZE, typename T, typename Ext>
void extend(T &obj, Ext &&ext) {
ext.deserialize(*this, _reader, obj, [](auto &s, auto &v) { s.template value<VSIZE>(v); });
static_assert(details::HasTValue<details::ExtensionTraits<Ext, T>>::value,
"this extension only supports overload with lambda");
ext.deserialize(*this, _reader, obj, [this](typename details::ExtensionTraits<Ext, T>::TValue &v) { value<VSIZE>(v); });
};
template<typename T, typename Ext>
void extend(T &obj, Ext &&ext) {
ext.deserialize(*this, _reader, obj, [](auto &s, auto &v) { s.object(v); });
static_assert(details::HasTValue<details::ExtensionTraits<Ext, T>>::value,
"this extension only supports overload with lambda");
ext.deserialize(*this, _reader, obj, [this](typename details::ExtensionTraits<Ext, T>::TValue &v) { object(v); });
};
/*
@@ -146,13 +150,13 @@ namespace bitsery {
* entropy overloads
*/
template<typename T, size_t N, typename Fnc>
void entropy(T &v, const T (&expectedValues)[N], Fnc &&fnc) {
void entropy(T &obj, const T (&expectedValues)[N], Fnc &&fnc) {
size_t index;
range(index, {{}, N + 1});
if (index)
v = expectedValues[index - 1];
obj = expectedValues[index - 1];
else
fnc(*this, v);
fnc(obj);
};
template<size_t VSIZE, typename T, size_t N>
@@ -166,13 +170,13 @@ namespace bitsery {
};
template<typename T, size_t N>
void entropy(T &v, const T (&expectedValues)[N]) {
void entropy(T &obj, const T (&expectedValues)[N]) {
size_t index;
range(index, {{}, N + 1});
if (index)
v = expectedValues[index - 1];
obj = expectedValues[index - 1];
else
object(v);
object(obj);
};
/*
@@ -181,10 +185,32 @@ namespace bitsery {
template<size_t VSIZE, typename T>
void text(T &str, size_t maxSize) {
static_assert(details::TextTraits<T>::isResizable,
"use text(T&) overload without `maxSize` for static containers");
size_t size;
readSize(size, maxSize);
str.resize(size);
details::TextTraits<T>::resize(str, size);
auto begin = std::begin(str);
auto end = std::next(begin, size);
procContainer<VSIZE>(begin, end, std::true_type{});
//null terminated character at the end
*end = {};
}
template<size_t VSIZE, typename T>
void text(T &str) {
static_assert(!details::TextTraits<T>::isResizable,
"use text(T&, size_t) overload with `maxSize` for dynamic containers");
size_t size;
auto begin = std::begin(str);
auto containerEnd = std::end(str);
assert(begin != containerEnd);
readSize(size, static_cast<size_t>(std::distance(begin, containerEnd) - 1));
//end of string, not en
auto end = std::next(begin, size);
procContainer<VSIZE>(std::begin(str), std::end(str), std::true_type{});
//null terminated character at the end
*end = {};
}
template<size_t VSIZE, typename T, size_t N>
@@ -205,45 +231,45 @@ namespace bitsery {
template<typename T, typename Fnc>
void container(T &&obj, size_t maxSize, Fnc &&fnc) {
static_assert(details::IsResizable<T>::value,
"use container(const T&) overload without `maxSize` for static containers");
decltype(obj.size()) size{};
static_assert(details::ContainerTraits<T>::isResizable,
"use container(T&) overload without `maxSize` for static containers");
size_t size{};
readSize(size, maxSize);
obj.resize(size);
details::ContainerTraits<T>::resize(obj, size);
procContainer(std::begin(obj), std::end(obj), std::forward<Fnc>(fnc));
}
template<size_t VSIZE, typename T>
void container(T &obj, size_t maxSize) {
static_assert(details::IsResizable<T>::value,
"use container(const T&) overload without `maxSize` for static containers");
decltype(obj.size()) size{};
static_assert(details::ContainerTraits<T>::isResizable,
"use container(T&) overload without `maxSize` for static containers");
size_t size{};
readSize(size, maxSize);
obj.resize(size);
details::ContainerTraits<T>::resize(obj, size);
procContainer<VSIZE>(std::begin(obj), std::end(obj), std::false_type{});
}
template<typename T>
void container(T &obj, size_t maxSize) {
static_assert(details::IsResizable<T>::value,
"use container(const T&) overload without `maxSize` for static containers");
decltype(obj.size()) size{};
static_assert(details::ContainerTraits<T>::isResizable,
"use container(T&) overload without `maxSize` for static containers");
size_t size{};
readSize(size, maxSize);
obj.resize(size);
details::ContainerTraits<T>::resize(obj, size);
procContainer(std::begin(obj), std::end(obj));
}
//fixed size containers
template<typename T, typename Fnc, typename std::enable_if<!std::is_integral<Fnc>::value>::type * = nullptr>
void container(T &&obj, Fnc &&fnc) {
static_assert(!details::IsResizable<T>::value,
static_assert(!details::ContainerTraits<T>::isResizable,
"use container(T&, size_t, Fnc) overload with `maxSize` for dynamic containers");
procContainer(std::begin(obj), std::end(obj), std::forward<Fnc>(fnc));
}
template<size_t VSIZE, typename T>
void container(T &obj) {
static_assert(!details::IsResizable<T>::value,
static_assert(!details::ContainerTraits<T>::isResizable,
"use container(T&, size_t) overload with `maxSize` for dynamic containers");
static_assert(VSIZE > 0);
procContainer<VSIZE>(std::begin(obj), std::end(obj), std::false_type{});
@@ -251,7 +277,7 @@ namespace bitsery {
template<typename T>
void container(T &obj) {
static_assert(!details::IsResizable<T>::value,
static_assert(!details::ContainerTraits<T>::isResizable,
"use container(T&, size_t) overload with `maxSize` for dynamic containers");
procContainer(std::begin(obj), std::end(obj));
}
@@ -304,16 +330,16 @@ namespace bitsery {
void extend8b(T &v, Ext &&ext) { extend<8>(v, std::forward<Ext>(ext)); };
template<typename T, size_t N>
void entropy1b(T &v, const T (&expectedValues)[N]) { entropy<1>(v, expectedValues); };
void entropy1b(T &v, const T (&expectedValues)[N]) { entropy<1, T, N>(v, expectedValues); };
template<typename T, size_t N>
void entropy2b(T &v, const T (&expectedValues)[N]) { entropy<2>(v, expectedValues); };
void entropy2b(T &v, const T (&expectedValues)[N]) { entropy<2, T, N>(v, expectedValues); };
template<typename T, size_t N>
void entropy4b(T &v, const T (&expectedValues)[N]) { entropy<4>(v, expectedValues); };
void entropy4b(T &v, const T (&expectedValues)[N]) { entropy<4, T, N>(v, expectedValues); };
template<typename T, size_t N>
void entropy8b(T &v, const T (&expectedValues)[N]) { entropy<8>(v, expectedValues); };
void entropy8b(T &v, const T (&expectedValues)[N]) { entropy<8, T, N>(v, expectedValues); };
template<typename T>
void text1b(T &str, size_t maxSize) { text<1>(str, maxSize); }
@@ -371,7 +397,7 @@ namespace bitsery {
void container8b(T (&arr)[N]) { container<8>(arr); }
private:
Reader &_reader;
BasicBufferReader<Config> &_reader;
void* _context;
void readSize(size_t &size, size_t maxSize) {
@@ -402,7 +428,7 @@ namespace bitsery {
template<typename It, typename Fnc>
void procContainer(It first, It last, Fnc fnc) {
for (; first != last; ++first)
fnc(*this, *first);
fnc(*first);
};
//process object types
@@ -414,6 +440,8 @@ namespace bitsery {
};
//helper type
using Deserializer = BasicDeserializer<DefaultConfig>;
}

52
include/bitsery/details/buffer_common.h
View File

@@ -23,7 +23,6 @@
#ifndef BITSERY_DETAILS_BUFFER_COMMON_H
#define BITSERY_DETAILS_BUFFER_COMMON_H
#include <type_traits>
#include <algorithm>
#include <utility>
#include <cassert>
@@ -31,6 +30,7 @@
#include <stack>
#include <cstring>
#include "both_common.h"
#include "traits.h"
namespace bitsery {
@@ -56,8 +56,11 @@ namespace bitsery {
};
namespace details {
template<typename T>
constexpr size_t BITS_SIZE = sizeof(T) << 3;
struct BITS_SIZE:public std::integral_constant<size_t, sizeof(T) << 3> {
};
//add swap functions to class, to avoid compilation warning about unused functions
struct swapImpl {
@@ -297,7 +300,6 @@ namespace bitsery {
auto sessionsIt = std::back_inserter(_sessions);
_pos = std::next(_end, -sessionsOffset);
while (std::distance(_pos, endSessionsSizesIt) > 0) {
//todo try to read into iterator directly
size_t size;
details::readSize(_reader, size);
*sessionsIt++ = size;
@@ -310,16 +312,17 @@ namespace bitsery {
}
};
template<typename Buffer, bool isFixed>
template<typename Buffer, bool isResizable>
class WriteBufferContext {
};
template<typename Buffer>
class WriteBufferContext<Buffer, true>{
class WriteBufferContext<Buffer, false>{
public:
using ValueType = typename Buffer::value_type;
using IteratorType = typename Buffer::iterator;
using ValueType = typename BufferContainerTraits<Buffer>::TValue;
using IteratorType = typename BufferContainerTraits<Buffer>::TIterator;
using DifferenceType = typename BufferContainerTraits<Buffer>::TDifference;
explicit WriteBufferContext(Buffer &buffer)
: _buffer{buffer},
@@ -329,7 +332,7 @@ namespace bitsery {
}
void write(const ValueType *data, size_t size) {
assert(std::distance(_outIt, _end) >= static_cast<typename Buffer::difference_type>(size));
assert(std::distance(_outIt, _end) >= static_cast<DifferenceType>(size));
memcpy(_outIt, data, size);
_outIt += size;
}
@@ -346,48 +349,47 @@ namespace bitsery {
};
template<typename Buffer>
class WriteBufferContext<Buffer, false> {
class WriteBufferContext<Buffer, true> {
public:
using ValueType = typename Buffer::value_type;
using IteratorType = typename Buffer::iterator;
using TValue = typename BufferContainerTraits<Buffer>::TValue;
using TIterator = typename BufferContainerTraits<Buffer>::TIterator;
using TDifference = typename BufferContainerTraits<Buffer>::TDifference;
explicit WriteBufferContext(Buffer &buffer)
: _buffer{buffer}
{
resizeToCapacity(0);
getIterators(0);
}
void write(const ValueType *data, size_t size) {
if ((_end - _outIt) >= static_cast<typename Buffer::difference_type>(size)) {
void write(const TValue *data, size_t size) {
if ((_end - _outIt) >= static_cast< TDifference >(size)) {
std::memcpy(_outIt, data, size);
_outIt += size;
} else {
//get current position before invalidating iterators
auto pos = std::distance(std::addressof(*std::begin(_buffer)), _outIt);
//make dummy call to back insert iterator to resize buffer
*(std::back_insert_iterator<Buffer>(_buffer)) = {};
resizeToCapacity(pos);
//increase container size
BufferContainerTraits<Buffer>::increaseBufferSize(_buffer);
//restore iterators
getIterators(pos);
write(data, size);
}
}
BufferRange<IteratorType> getWrittenRange() const {
BufferRange<TIterator> getWrittenRange() const {
auto begin = std::begin(_buffer);
return BufferRange<IteratorType>{begin, std::next(begin, _outIt - std::addressof(*begin))};
return BufferRange<TIterator>{begin, std::next(begin, _outIt - std::addressof(*begin))};
}
private:
void resizeToCapacity(typename Buffer::difference_type writePos) {
if (_buffer.capacity() != _buffer.size()) {
_buffer.resize(_buffer.capacity());
}
void getIterators(TDifference writePos) {
_end = std::addressof(*std::end(_buffer));
_outIt = std::addressof(*std::next(std::begin(_buffer), writePos));
}
Buffer &_buffer;
ValueType* _outIt;
ValueType* _end;
TValue* _outIt;
TValue* _end;
};
}

54
include/bitsery/details/serialization_common.h
View File

@@ -33,17 +33,17 @@ namespace bitsery {
template<typename T, typename Enable = void>
struct SAME_SIZE_UNSIGNED_TYPE {
typedef std::make_unsigned_t<T> type;
using type = typename std::make_unsigned<T>::type;
};
template<typename T>
struct SAME_SIZE_UNSIGNED_TYPE<T, typename std::enable_if<std::is_enum<T>::value>::type> {
typedef std::make_unsigned_t<std::underlying_type_t<T>> type;
using type = typename std::make_unsigned<typename std::underlying_type<T>::type>::type;
};
template<typename T>
struct SAME_SIZE_UNSIGNED_TYPE<T, typename std::enable_if<std::is_floating_point<T>::value>::type> {
typedef std::conditional_t<std::is_same<T, float>::value, uint32_t, uint64_t> type;
using type = typename std::conditional<std::is_same<T, float>::value, uint32_t, uint64_t>::type;
};
template<typename T>
@@ -60,20 +60,8 @@ namespace bitsery {
return getSize(max - min, 0);
}
template <typename T, typename = int>
struct IsResizable : std::false_type {};
template <typename T>
struct IsResizable <T, decltype((void)std::declval<T>().resize(1u), 0)> : std::true_type {};
}
/*
* serialization/deserialization context
*/
struct Context {
void* getCustomPtr();
};
/*
* range functions in bitsery namespace because these are used by user
*/
@@ -100,8 +88,8 @@ namespace bitsery {
min{minValue},
max{maxValue},
bitsRequired{details::calcRequiredBits(
static_cast<std::underlying_type_t<T>>(min),
static_cast<std::underlying_type_t<T>>(max))} {
static_cast<typename std::underlying_type<T>::type>(min),
static_cast<typename std::underlying_type<T>::type>(max))} {
}
const T min;
@@ -144,18 +132,18 @@ namespace bitsery {
*/
template<typename T, typename std::enable_if<std::is_integral<T>::value>::type * = nullptr>
auto getRangeValue(const T &v, const RangeSpec<T> &r) {
SAME_SIZE_UNSIGNED<T> getRangeValue(const T &v, const RangeSpec<T> &r) {
return static_cast<SAME_SIZE_UNSIGNED<T>>(v - r.min);
};
template<typename T, typename std::enable_if<std::is_enum<T>::value>::type * = nullptr>
auto getRangeValue(const T &v, const RangeSpec<T> &r) {
SAME_SIZE_UNSIGNED<T> getRangeValue(const T &v, const RangeSpec<T> &r) {
using VT = SAME_SIZE_UNSIGNED<T>;
return static_cast<VT>(static_cast<VT>(v) - static_cast<VT>(r.min));
};
template<typename T, typename std::enable_if<std::is_floating_point<T>::value>::type * = nullptr>
auto getRangeValue(const T &v, const RangeSpec<T> &r) {
SAME_SIZE_UNSIGNED<T> getRangeValue(const T &v, const RangeSpec<T> &r) {
using VT = SAME_SIZE_UNSIGNED<T>;
const VT maxUint = (static_cast<VT>(1) << r.bitsRequired) - 1;
const auto ratio = (v - r.min) / (r.max - r.min);
@@ -169,7 +157,7 @@ namespace bitsery {
template<typename T, typename std::enable_if<std::is_enum<T>::value>::type * = nullptr>
void setRangeValue(T &v, const RangeSpec<T> &r) {
using VT = std::underlying_type_t<T>;
using VT = typename std::underlying_type<T>::type;
reinterpret_cast<VT &>(v) += static_cast<VT>(r.min);
};
@@ -188,7 +176,7 @@ namespace bitsery {
template<typename T, typename std::enable_if<std::is_enum<T>::value>::type * = nullptr>
bool isRangeValid(const T &v, const RangeSpec<T> &r) {
using VT = std::underlying_type_t<T>;
using VT = typename std::underlying_type<T>::type;
return !(static_cast<VT>(r.min) > static_cast<VT>(v)
|| static_cast<VT>(v) > static_cast<VT>(r.max));
}
@@ -215,7 +203,13 @@ namespace bitsery {
template<typename S, typename T, typename Enabled = void>
struct SerializeFunction {
static void invoke(S &s, T &v) {
static_assert(!std::is_void<Enabled>::value, "please define 'serialize' function.");
static_assert(!std::is_void<Enabled>::value,
"\nPlease define 'serialize' function for your type:\n"
" template<typename S>\n"
" void serialize(S& s, <YourType>& o)\n"
" {\n"
" ...\n"
" }\n");
}
};
@@ -228,6 +222,20 @@ namespace bitsery {
}
};
/**
* used to check if extension supports overloads with `object` and `value<N>`
*/
template <typename T, typename Enable = void>
struct HasTValue:public std::false_type {
};
template <typename T>
struct HasTValue<T, typename std::enable_if<
//only works when TValue is defined, and is not void
!std::is_same<void, typename T::TValue>::value
>::type>: public std::true_type {
};
/*
* delta functions

144
include/bitsery/details/traits.h Normal file
View File

@@ -0,0 +1,144 @@
//MIT License
//
//Copyright (c) 2017 Mindaugas Vinkelis
//
//Permission is hereby granted, free of charge, to any person obtaining a copy
//of this software and associated documentation files (the "Software"), to deal
//in the Software without restriction, including without limitation the rights
//to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
//copies of the Software, and to permit persons to whom the Software is
//furnished to do so, subject to the following conditions:
//
//The above copyright notice and this permission notice shall be included in all
//copies or substantial portions of the Software.
//
//THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
//IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
//FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
//AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
//LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
//OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
//SOFTWARE.
#ifndef BITSERY_DETAILS_TRAITS_H
#define BITSERY_DETAILS_TRAITS_H
#include <type_traits>
#include <string>
namespace bitsery {
namespace details {
/*
* helper traits that is used internaly, or by other traits
*/
template <typename T, typename = int>
struct IsResizable : std::false_type {};
template <typename T>
struct IsResizable <T, decltype((void)std::declval<T>().resize(1u), 0)> : std::true_type {};
/*
* core library traits, used to extend library for custom types
*/
//traits for extension
template<typename Extension, typename T>
struct ExtensionTraits {
//this type is used, when using extesion without custom lambda
// eg.: extension4b>(obj, myextension{}) will call s.value4b(obj) for TValue
// or extesion(obj, myextension{}) will call s.object(obj) for TValue
//if this is not defined, then these functions are disabled
using TValue = void;
};
//traits for containers
template<typename T>
struct ContainerTraits {
//default behaviour is resizable if container has method T::resize(size_t)
static constexpr bool isResizable = IsResizable<T>::value;
//resize function, called only if container is resizable
static void resize(T& container, size_t size) {
container.resize(size);
}
//get container size
static size_t size(const T& container) {
return container.size();
}
};
//traits for text
template<typename T>
struct TextTraits {
static constexpr bool isResizable = true;
//resize is without null-terminated character as with std::string,
//but null terminated character will always be written
//if you container doesn't add null-terminated character automaticaly, resize it to size+1;
static void resize(T& container, size_t size) {
container.resize(size);
}
//used for serialization to get text length
//length is until null-terminated character, size and length might not be equal
static size_t length(const T& container) {
auto begin = std::begin(container);
using TValue = typename std::decay<decltype(*begin)>::type;
return std::char_traits<TValue>::length(std::addressof(*begin));
}
};
//text traits specialization for std::string
//for std::string return length as size(), for faster performance, so we don't need to traverse string to find null-terminated characeter
//although it is not correct behaviour, meaning that string might have null-terminated characters in the middle,
//but in this case it your decision if you store buffer in string and serialize it as a text.
template<typename ... Args>
struct TextTraits<std::basic_string<Args...>> {
static constexpr bool isResizable = true;
//resize is without null-terminated character as with std::string,
//but null terminated character will always be written
//if you container doesn't add null-terminated character automaticaly, resize it to size+1;
static void resize(std::basic_string<Args...>& container, size_t size) {
container.resize(size);
}
//used for serialization to get text length
//length is until null-terminated character, size and length might not be equal
static size_t length(const std::basic_string<Args...>& container) {
return container.size();
}
};
//traits only for buffer reader/writer
template <typename T>
struct BufferContainerTraits: public ContainerTraits<T> {
//this function is only used by BufferWriter, when writing data to buffer,
//it is called only current buffer size is not enough to write.
//it is used to dramaticaly improve performance by updating buffer directly
//instead of using back_insert_iterator to append each byte to buffer.
//thats why BufferWriter return range iterators
static void increaseBufferSize(T& container) {
//use default implementation behaviour;
//call push_back to use default resize strategy
container.push_back({});
//after allocation resize to take all capacity
container.resize(container.capacity());
}
using TValue = typename T::value_type;
using TDifference = typename T::difference_type;
using TIterator = typename T::iterator;
};
}
}
#endif //BITSERY_DETAILS_TRAITS_H

73
include/bitsery/ext/container_map.h Normal file
View File

@@ -0,0 +1,73 @@
//MIT License
//
//Copyright (c) 2017 Mindaugas Vinkelis
//
//Permission is hereby granted, free of charge, to any person obtaining a copy
//of this software and associated documentation files (the "Software"), to deal
//in the Software without restriction, including without limitation the rights
//to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
//copies of the Software, and to permit persons to whom the Software is
//furnished to do so, subject to the following conditions:
//
//The above copyright notice and this permission notice shall be included in all
//copies or substantial portions of the Software.
//
//THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
//IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
//FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
//AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
//LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
//OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
//SOFTWARE.
#ifndef BITSERY_EXT_CONTAINER_MAP_H
#define BITSERY_EXT_CONTAINER_MAP_H
namespace bitsery {
namespace ext {
class containerMap {
public:
template<typename Ser, typename Writer, typename T, typename Fnc>
void serialize(Ser &, Writer &writer, const T &obj, Fnc &&fnc) const {
using TKey = typename T::key_type;
using TValue = typename T::mapped_type;
details::writeSize(writer, obj.size());
for (auto& v:obj)
fnc(const_cast<TKey&>(v.first), const_cast<TValue&>(v.second));
}
template<typename Des, typename Reader, typename T, typename Fnc>
void deserialize(Des &, Reader& reader, T &obj, Fnc &&fnc) const {
using TKey = typename T::key_type;
using TValue = typename T::mapped_type;
size_t size{};
details::readSize(reader, size);
auto hint = obj.begin();
obj.clear();
for (auto i = 0u; i < size; ++i) {
TKey key;
TValue value;
fnc(key, value);
hint = obj.emplace_hint(hint, std::move(key), std::move(value));
}
}
};
}
namespace details {
template <typename T>
struct ExtensionTraits<ext::containerMap, T> {
//do not define TValue, because we dont have default behaviour for this key+value
};
}
}
#endif //BITSERY_EXT_CONTAINER_MAP_H

16
include/bitsery/ext/optional.h
View File

@@ -32,6 +32,8 @@
// template <typename T>
// using optional = experimental::optional<T>;
//}
#include <type_traits>
namespace bitsery {
namespace ext {
@@ -40,6 +42,7 @@ namespace bitsery {
class optional {
public:
template<typename T>
constexpr void assertType() const {
using TOpt = typename std::remove_cv<T>::type;
@@ -53,7 +56,7 @@ namespace bitsery {
assertType<T>();
ser.boolByte(static_cast<bool>(obj));
if (obj)
fnc(ser, *obj);
fnc(const_cast<typename T::value_type& >(*obj));
}
template<typename Des, typename Reader, typename T, typename Fnc>
@@ -63,7 +66,7 @@ namespace bitsery {
des.boolByte(exists);
if (exists) {
typename T::value_type tmp{};
fnc(des, tmp);
fnc(tmp);
obj = tmp;
} else {
//experimental optional doesnt have .reset method
@@ -71,8 +74,15 @@ namespace bitsery {
}
}
};
}
namespace details {
template <typename T>
struct ExtensionTraits<ext::optional, T> {
using TValue = typename T::value_type;
};
}
}

135
include/bitsery/serializer.h
View File

@@ -31,10 +31,13 @@
namespace bitsery {
template<typename Writter>
class Serializer {
template<typename Config>
class BasicSerializer {
public:
Serializer(Writter &w, void* context = nullptr) : _writter{w}, _context{context} {};
explicit BasicSerializer(BasicBufferWriter<Config> &w, void* context = nullptr) : _writter{w}, _context{context} {};
/*
* get serialization context.
@@ -48,13 +51,13 @@ namespace bitsery {
* object function
*/
template<typename T>
void object(T &&obj) {
details::SerializeFunction<Serializer, T>::invoke(*this, std::forward<T>(obj));
void object(const T &obj) {
details::SerializeFunction<BasicSerializer, T>::invoke(*this, const_cast<T& >(obj));
}
template<typename T, typename Fnc>
void object(T &&obj, Fnc &&fnc) {
fnc(*this, std::forward<T>(obj));
void object(const T &obj, Fnc &&fnc) {
fnc(const_cast<T& >(obj));
};
/*
@@ -69,7 +72,7 @@ namespace bitsery {
template<size_t VSIZE, typename T, typename std::enable_if<std::is_enum<T>::value>::type * = nullptr>
void value(const T &v) {
_writter.template writeBytes<VSIZE>(reinterpret_cast<const std::underlying_type_t<T> &>(v));
_writter.template writeBytes<VSIZE>(reinterpret_cast<const typename std::underlying_type<T>::type &>(v));
}
template<size_t VSIZE, typename T, typename std::enable_if<std::is_integral<T>::value>::type * = nullptr>
@@ -84,7 +87,7 @@ namespace bitsery {
template<typename T, typename Fnc>
void growable(const T &obj, Fnc &&fnc) {
_writter.beginSession();
fnc(*this, obj);
fnc(const_cast<T&>(obj));
_writter.endSession();
};
@@ -95,18 +98,20 @@ namespace bitsery {
template<typename T, typename Ext, typename Fnc>
void extend(const T &obj, Ext &&ext, Fnc &&fnc) {
ext.serialize(*this, _writter, obj, std::forward<Fnc>(fnc));
};
template<size_t VSIZE, typename T, typename Ext>
void extend(const T &obj, Ext &&ext) {
ext.serialize(*this, _writter, obj, [](auto &s, auto &v) { s.template value<VSIZE>(v); });
static_assert(details::HasTValue<details::ExtensionTraits<Ext, T>>::value,
"this extension only supports overload with lambda");
ext.serialize(*this, _writter, obj, [this](typename details::ExtensionTraits<Ext, T>::TValue &v) { value<VSIZE>(v); });
};
template<typename T, typename Ext>
void extend(const T &obj, Ext &&ext) {
ext.serialize(*this, _writter, obj, [](auto &s, auto &v) { s.object(v); });
static_assert(details::HasTValue<details::ExtensionTraits<Ext, T>>::value,
"this extension only supports overload with lambda");
ext.serialize(*this, _writter, obj, [this](typename details::ExtensionTraits<Ext, T>::TValue &v) { object(v); });
};
/*
@@ -136,11 +141,11 @@ namespace bitsery {
* entropy overloads
*/
template<typename T, size_t N, typename Fnc>
void entropy(const T &v, const T (&expectedValues)[N], Fnc &&fnc) {
auto index = details::findEntropyIndex(v, expectedValues);
void entropy(const T &obj, const T (&expectedValues)[N], Fnc &&fnc) {
auto index = details::findEntropyIndex(obj, expectedValues);
range(index, {{}, N + 1});
if (!index)
fnc(*this, v);
fnc(const_cast<T&>(obj));
};
template<size_t VSIZE, typename T, size_t N>
@@ -152,11 +157,11 @@ namespace bitsery {
};
template<typename T, size_t N>
void entropy(const T &v, const T (&expectedValues)[N]) {
auto index = details::findEntropyIndex(v, expectedValues);
void entropy(const T &obj, const T (&expectedValues)[N]) {
auto index = details::findEntropyIndex(obj, expectedValues);
range(index, {{}, N + 1});
if (!index)
object(v);
object(obj);
};
/*
@@ -165,20 +170,37 @@ namespace bitsery {
template<size_t VSIZE, typename T>
void text(const T &str, size_t maxSize) {
auto first = std::begin(str);
auto last = std::end(str);
auto size = static_cast<size_t>(std::distance(first, last));
static_assert(details::TextTraits<T>::isResizable,
"use text(const T&) overload without `maxSize` for static container");
auto size = details::TextTraits<T>::length(str);
//size can be equal to maxSize
assert(size <= maxSize);
writeSize(size);
procContainer<VSIZE>(first, last, std::true_type{});
details::writeSize(_writter, size);
auto begin = std::begin(str);
procContainer<VSIZE>(begin, std::next(begin, size), std::true_type{});
}
template<size_t VSIZE, typename T>
void text(const T &str) {
static_assert(!details::TextTraits<T>::isResizable,
"use text(const T&, size_t) overload with `maxSize` for dynamic containers");
auto size = details::TextTraits<T>::length(str);
auto begin = std::begin(str);
auto end = std::end(str);
//size must be less than container capacity, because we need to store null-terminated character
assert(size < std::distance(begin, end));
details::writeSize(_writter, size);
procContainer<VSIZE>(begin, std::next(begin, size), std::true_type{});
}
template<size_t VSIZE, typename T, size_t N>
void text(const T (&str)[N]) {
auto first = std::begin(str);
auto last = std::next(first, std::min(std::char_traits<T>::length(str), N - 1));
writeSize(std::distance(first, last));
procContainer<VSIZE>(first, last, std::true_type{});
auto size = details::TextTraits<T[N]>::length(str);
assert(size < N);
details::writeSize(_writter, size);
auto begin = std::begin(str);
procContainer<VSIZE>(begin, std::next(begin, size), std::true_type{});
}
/*
@@ -189,30 +211,34 @@ namespace bitsery {
template<typename T, typename Fnc>
void container(const T &obj, size_t maxSize, Fnc &&fnc) {
static_assert(details::IsResizable<T>::value,
static_assert(details::ContainerTraits<T>::isResizable,
"use container(const T&, Fnc) overload without `maxSize` for static containers");
assert(obj.size() <= maxSize);
writeSize(obj.size());
auto size = details::ContainerTraits<T>::size(obj);
assert(size <= maxSize);
details::writeSize(_writter, size);
procContainer(std::begin(obj), std::end(obj), std::forward<Fnc>(fnc));
}
template<size_t VSIZE, typename T>
void container(const T &obj, size_t maxSize) {
static_assert(details::IsResizable<T>::value,
static_assert(details::ContainerTraits<T>::isResizable,
"use container(const T&) overload without `maxSize` for static containers");
static_assert(VSIZE > 0, "");
assert(obj.size() <= maxSize);
writeSize(obj.size());
auto size = details::ContainerTraits<T>::size(obj);
assert(size <= maxSize);
details::writeSize(_writter, size);
//todo optimisation is possible for contigous containers, but currently there is no compile-time check for this
procContainer<VSIZE>(std::begin(obj), std::end(obj), std::false_type{});
}
template<typename T>
void container(const T &obj, size_t maxSize) {
static_assert(details::IsResizable<T>::value,
static_assert(details::ContainerTraits<T>::isResizable,
"use container(const T&) overload without `maxSize` for static containers");
assert(obj.size() <= maxSize);
writeSize(obj.size());
auto size = details::ContainerTraits<T>::size(obj);
assert(size <= maxSize);
details::writeSize(_writter, size);
procContainer(std::begin(obj), std::end(obj));
}
@@ -220,14 +246,14 @@ namespace bitsery {
template<typename T, typename Fnc, typename std::enable_if<!std::is_integral<Fnc>::value>::type * = nullptr>
void container(const T &obj, Fnc &&fnc) {
static_assert(!details::IsResizable<T>::value,
static_assert(!details::ContainerTraits<T>::isResizable,
"use container(const T&, size_t, Fnc) overload with `maxSize` for dynamic containers");
procContainer(std::begin(obj), std::end(obj), std::forward<Fnc>(fnc));
}
template<size_t VSIZE, typename T>
void container(const T &obj) {
static_assert(!details::IsResizable<T>::value,
static_assert(!details::ContainerTraits<T>::isResizable,
"use container(const T&, size_t) overload with `maxSize` for dynamic containers");
static_assert(VSIZE > 0, "");
//todo optimisation is possible for contigous containers, but currently there is no compile-time check for this
@@ -236,7 +262,7 @@ namespace bitsery {
template<typename T>
void container(const T &obj) {
static_assert(!details::IsResizable<T>::value,
static_assert(!details::ContainerTraits<T>::isResizable,
"use container(const T&, size_t) overload with `maxSize` for dynamic containers");
procContainer(std::begin(obj), std::end(obj));
}
@@ -291,22 +317,22 @@ namespace bitsery {
template<typename T, size_t N>
void entropy1b(const T &v, const T (&expectedValues)[N]) {
entropy<1>(v, expectedValues);
entropy<1, T, N>(v, expectedValues);
};
template<typename T, size_t N>
void entropy2b(const T &v, const T (&expectedValues)[N]) {
entropy<2>(v, expectedValues);
entropy<2, T, N>(v, expectedValues);
};
template<typename T, size_t N>
void entropy4b(const T &v, const T (&expectedValues)[N]) {
entropy<4>(v, expectedValues);
entropy<4, T, N>(v, expectedValues);
};
template<typename T, size_t N>
void entropy8b(const T &v, const T (&expectedValues)[N]) {
entropy<8>(v, expectedValues);
entropy<8, T, N>(v, expectedValues);
};
template<typename T>
@@ -319,13 +345,13 @@ namespace bitsery {
void text4b(const T &str, size_t maxSize) { text<4>(str, maxSize); }
template<typename T, size_t N>
void text1b(const T (&str)[N]) { text<1>(str); }
void text1b(const T (&str)[N]) { text<1, T, N>(str); }
template<typename T, size_t N>
void text2b(const T (&str)[N]) { text<2>(str); }
void text2b(const T (&str)[N]) { text<2, T, N>(str); }
template<typename T, size_t N>
void text4b(const T (&str)[N]) { text<4>(str); }
void text4b(const T (&str)[N]) { text<4, T, N>(str); }
template<typename T>
void container1b(T &&obj, size_t maxSize) { container<1>(std::forward<T>(obj), maxSize); }
@@ -364,13 +390,9 @@ namespace bitsery {
void container8b(const T (&arr)[N]) { container<8>(arr); }
private:
Writter &_writter;
BasicBufferWriter<Config> &_writter;
void* _context;
void writeSize(const size_t size) {
details::writeSize(_writter, size);
}
//process value types
//false_type means that we must process all elements individually
template<size_t VSIZE, typename It>
@@ -389,8 +411,10 @@ namespace bitsery {
//process by calling functions
template<typename It, typename Fnc>
void procContainer(It first, It last, Fnc fnc) {
for (; first != last; ++first)
fnc(*this, *first);
using TValue = typename std::decay<decltype(*first)>::type;
for (; first != last; ++first) {
fnc(const_cast<TValue&>(*first));
}
};
//process object types
@@ -402,5 +426,8 @@ namespace bitsery {
};
//helper type
using Serializer = BasicSerializer<DefaultConfig>;
}
#endif //BITSERY_SERIALIZER_H

3
tests/buffer_endianness.cpp
View File

@@ -40,7 +40,6 @@ constexpr EndiannessType getInverseEndianness(EndiannessType e) {
struct InverseEndiannessConfig {
static constexpr bitsery::EndiannessType NetworkEndianness = getInverseEndianness(DefaultConfig::NetworkEndianness);
static constexpr bool FixedBufferSize = DefaultConfig::FixedBufferSize;
using BufferType = DefaultConfig::BufferType;
};
@@ -147,7 +146,7 @@ struct IntegralUnsignedTypes {
TEST(BufferEndianness, WhenBufferValueTypeIs1ByteThenBitOperationsIsNotAffectedByEndianness) {
//fill initial values
static_assert(sizeof(DefaultConfig::BufferType::value_type) == 1, "currently only 1 byte size, value size is supported");
static_assert(sizeof(bitsery::details::BufferContainerTraits<DefaultConfig::BufferType>::TValue) == 1, "currently only 1 byte size, value size is supported");
//fill initial values
constexpr IntegralUnsignedTypes src {
0x0000334455667788,//bits 19

25
tests/buffer_operations.cpp
View File

@@ -47,6 +47,31 @@ constexpr size_t getBits(T v) {
// *** bits operations
TEST(BufferBitsAndBytesOperations, WriteAndReadBitsMaxTypeValues) {
Buffer buf;
BufferWriter bw{buf};
bw.writeBits(std::numeric_limits<uint64_t>::max(), 64);
bw.writeBits(std::numeric_limits<uint32_t>::max(), 32);
bw.writeBits(std::numeric_limits<uint16_t>::max(), 16);
bw.writeBits(std::numeric_limits<uint8_t>::max(), 8);
bw.flush();
BufferReader br{bw.getWrittenRange()};
uint64_t v64{};
uint32_t v32{};
uint16_t v16{};
uint8_t v8{};
br.readBits(v64, 64);
br.readBits(v32, 32);
br.readBits(v16, 16);
br.readBits(v8, 8);
EXPECT_THAT(v64, Eq(std::numeric_limits<uint64_t>::max()));
EXPECT_THAT(v32, Eq(std::numeric_limits<uint32_t>::max()));
EXPECT_THAT(v16, Eq(std::numeric_limits<uint16_t>::max()));
EXPECT_THAT(v8, Eq(std::numeric_limits<uint8_t>::max()));
}
TEST(BufferBitsAndBytesOperations, WriteAndReadBits) {
//setup data
constexpr IntegralUnsignedTypes data{

2
tests/buffer_writing.cpp
View File

@@ -33,13 +33,11 @@ using Buffer = bitsery::DefaultConfig::BufferType;
struct FixedBufferConfig {
static constexpr bitsery::EndiannessType NetworkEndianness = DefaultConfig::NetworkEndianness;
static constexpr bool FixedBufferSize = true;
using BufferType = std::array<uint8_t, 100>;
};
struct NonFixedBufferConfig {
static constexpr bitsery::EndiannessType NetworkEndianness = DefaultConfig::NetworkEndianness;
static constexpr bool FixedBufferSize = false;
using BufferType = std::vector<uint8_t>;
};

37
tests/serialization_container.cpp
View File

@@ -101,22 +101,21 @@ TYPED_TEST(SerializeContainerDynamicSizeArthmeticTypes, Values) {
TYPED_TEST(SerializeContainerDynamicSizeArthmeticTypes, CustomFunctionIncrements) {
SerializationContext ctx{};
using TValue = typename TestFixture::TValue;
auto ser = ctx.createSerializer();
ser.container(this->src, 1000, [](auto &s, auto v) {
//increment by 1 before writing
v++;
s.template value<sizeof(v)>(v);
ser.container(this->src, 1000, [&ser](TValue& v) {
ser.template value<sizeof(v)>(v);
});
auto des = ctx.createDeserializer();
des.container(this->res, 1000, [](auto &s, auto &v) {
s.template value<sizeof(v)>(v);
des.container(this->res, 1000, [&des](TValue &v) {
des.template value<sizeof(v)>(v);
//increment by 1 after reading
v++;
});
//decrement result by 2, before comparing for eq
//decrement result by 1, before comparing for eq
for (auto &v:this->res)
v -= 2;
v -= 1;
EXPECT_THAT(ctx.getBufferSize(), Eq(this->getExpectedBufSize(ctx)));
EXPECT_THAT(this->res, ContainerEq(this->src));
@@ -157,9 +156,9 @@ TYPED_TEST(SerializeContainerDynamicSizeCompositeTypes, DefaultSerializeFunction
TYPED_TEST(SerializeContainerDynamicSizeCompositeTypes, CustomFunctionThatDoNothing) {
SerializationContext ctx{};
using TValue = typename TestFixture::TValue;
auto emptyFnc = [](auto &s, auto &v) {};
auto emptyFnc = [](TValue &v) {};
ctx.createSerializer().container(this->src, 1000, emptyFnc);
ctx.createDeserializer().container(this->res, 1000, emptyFnc);
@@ -204,8 +203,7 @@ class SerializeContainerFixedSizeCompositeTypes : public SerializeContainerFixed
};
using StaticContainersWithCompositeTypes = ::testing::Types<
std::array<MyStruct1, 4>,
MyStruct1[4]>;
std::array<MyStruct1, 4>, MyStruct1[4]>;
TYPED_TEST_CASE(SerializeContainerFixedSizeCompositeTypes, StaticContainersWithCompositeTypes);
@@ -227,18 +225,20 @@ TYPED_TEST(SerializeContainerFixedSizeCompositeTypes, CustomFunctionThatSerializ
Container src{MyStruct1{0, 1}, MyStruct1{2, 3}, MyStruct1{4, 5}, MyStruct1{5134, 1532}};
Container res{};
using TValue = decltype(*std::begin(res));
SerializationContext ctx;
auto ser = ctx.createSerializer();
ser.container(src, [](auto &s, auto &v) {
ser.container(src, [&ser](TValue &v) {
char tmp{};
s.object(v);
s.value1b(tmp);
ser.object(v);
ser.value1b(tmp);
});
auto des = ctx.createDeserializer();
des.container(res, [](auto &s, auto &v) {
des.container(res, [&des](TValue &v) {
char tmp{};
s.object(v);
s.value1b(tmp);
des.object(v);
des.value1b(tmp);
});
EXPECT_THAT(ctx.getBufferSize(), Eq(this->getContainerSize() * (MyStruct1::SIZE + sizeof(char))));
@@ -246,4 +246,3 @@ TYPED_TEST(SerializeContainerFixedSizeCompositeTypes, CustomFunctionThatSerializ
}

16
tests/serialization_entropy.cpp
View File

@@ -101,16 +101,16 @@ TEST(SerializeEntropyEncoding, CustomFunctionNotEntropyEncoded) {
auto ser = ctx.createSerializer();
//lambdas differ only in capture clauses, it would make sense to use std::bind, but debugger crashes when it sees std::bind...
auto serLambda = [rangeForValue](auto& s, const MyStruct1& v) {
s.range(v.i1, rangeForValue);
s.range(v.i2, rangeForValue);
auto serLambda = [&ser, rangeForValue](MyStruct1& v) {
ser.range(v.i1, rangeForValue);
ser.range(v.i2, rangeForValue);
};
ser.entropy(v, entropyValues, serLambda);
auto des = ctx.createDeserializer();
auto desLambda = [rangeForValue](auto& s, MyStruct1& v) {
s.range(v.i1, rangeForValue);
s.range(v.i2, rangeForValue);
auto desLambda = [&des, rangeForValue](MyStruct1& v) {
des.range(v.i1, rangeForValue);
des.range(v.i2, rangeForValue);
};
des.entropy(res, entropyValues, desLambda);
@@ -127,8 +127,8 @@ TEST(SerializeEntropyEncoding, WhenEntropyEncodedThenCustomFunctionNotInvoked) {
MyStruct1{4849,89}, MyStruct1{0,1}};
SerializationContext ctx;
ctx.createSerializer().entropy(v, entropyValues, [](bitsery::Serializer<bitsery::BufferWriter>& ,const MyStruct1& ) {});
ctx.createDeserializer().entropy(res, entropyValues, [](bitsery::Deserializer<bitsery::BufferReader>&, MyStruct1& ) {});
ctx.createSerializer().entropy(v, entropyValues, [](MyStruct1& ) {});
ctx.createDeserializer().entropy(res, entropyValues, []( MyStruct1& ) {});
EXPECT_THAT(res, Eq(v));
EXPECT_THAT(ctx.getBufferSize(), Eq(1));

138
tests/serialization_ext_container_map.cpp Normal file
View File

@@ -0,0 +1,138 @@
//MIT License
//
//Copyright (c) 2017 Mindaugas Vinkelis
//
//Permission is hereby granted, free of charge, to any person obtaining a copy
//of this software and associated documentation files (the "Software"), to deal
//in the Software without restriction, including without limitation the rights
//to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
//copies of the Software, and to permit persons to whom the Software is
//furnished to do so, subject to the following conditions:
//
//The above copyright notice and this permission notice shall be included in all
//copies or substantial portions of the Software.
//
//THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
//IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
//FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
//AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
//LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
//OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
//SOFTWARE.
#include <gmock/gmock.h>
#include "serialization_test_utils.h"
#include <bitsery/ext/container_map.h>
#include <unordered_map>
#include <map>
using containerMap = bitsery::ext::containerMap;
using testing::Eq;
template<typename Container>
Container createData() {
return {};
}
template<>
std::unordered_map<std::string, MyStruct1> createData<std::unordered_map<std::string, MyStruct1>>() {
return {
std::make_pair("some key", MyStruct1{874,456}),
std::make_pair("other key", MyStruct1{-34,8645}),
std::make_pair("secret key", MyStruct1{-4878,3468975})
};
}
template<>
std::unordered_map<int32_t, float> createData<std::unordered_map<int32_t, float>>() {
return {
std::pair<int32_t , float>(545, 45.485f),
std::pair<int32_t , float>(6748, -7891.5f),
std::pair<int32_t , float>(845, -457.0f)
};
}
template<>
std::map<MyEnumClass, MyStruct1> createData<std::map<MyEnumClass, MyStruct1>>() {
return {
std::make_pair(MyEnumClass::E3, MyStruct1{874,456}),
std::make_pair(MyEnumClass::E6, MyStruct1{-34,8645}),
std::make_pair(MyEnumClass::E2, MyStruct1{-4878,3468975})
};
}
template<>
std::map<int32_t ,int64_t> createData<std::map<int32_t ,int64_t>>() {
return {//these are optimized with range and entropy
std::pair<int32_t, int64_t>(-45, -984196845ll),
std::pair<int32_t, int64_t>(54, 1ll),
std::pair<int32_t, int64_t>(98, 3ll)
};
}
template<typename T>
class SerializeExtensionContainerMap : public testing::Test {
public:
using TContainer = T;
const TContainer src = createData<TContainer>();
TContainer res{};
};
using SerializeExtensionContainerMapTypes = ::testing::Types<
std::unordered_map<std::string, MyStruct1>,
std::unordered_map<int32_t, float>,
std::map<MyEnumClass , MyStruct1>,
std::map<int32_t ,int64_t>
>;
TYPED_TEST_CASE(SerializeExtensionContainerMap, SerializeExtensionContainerMapTypes);
namespace bitsery {
template <typename S>
void serialize(S& s, std::unordered_map<std::string, MyStruct1>& o) {
s.extend(o, containerMap{}, [&s](std::string& key, MyStruct1& value) {
s.text1b(key, 100);
s.object(value);
});
}
template <typename S>
void serialize(S& s, std::unordered_map<int32_t, float>& o) {
s.extend(o, containerMap{}, [&s](int32_t& key, float& value) {
s.value4b(key);
s.value4b(value);
});
}
template <typename S>
void serialize(S& s, std::map<MyEnumClass , MyStruct1>& o) {
s.extend(o, containerMap{}, [&s](MyEnumClass& key, MyStruct1& value) {
s.value4b(key);
s.object(value);
});
}
template <typename S>
void serialize(S& s, std::map<int32_t ,int64_t>& o) {
s.extend(o, containerMap{}, [&s](int32_t& key, int64_t& value) {
s.range(key, bitsery::RangeSpec<int32_t>{-100,100});
constexpr int64_t ev[3]{1ll, 2ll, 3ll};
s.entropy8b(value, ev);
});
}
}
TYPED_TEST(SerializeExtensionContainerMap, SerializeAndDeserializeEquals) {
SerializationContext ctx1;
ctx1.createSerializer().object(this->src);
ctx1.createDeserializer().object(this->res);
EXPECT_THAT(this->res, Eq(this->src));
}

7
tests/serialization_growable.cpp
View File

@@ -26,7 +26,7 @@
using namespace testing;
using Buffer = typename bitsery::DefaultConfig::BufferType;
using DiffType = typename Buffer::difference_type;
using DiffType = typename bitsery::details::BufferContainerTraits<Buffer>::TDifference;
struct DataV1 {
int32_t v1;
@@ -46,7 +46,7 @@ struct DataV3 {
TEST(SerializeGrowable, WriteSessionsDataAtBufferEndAfterFlush) {
SerializationContext ctx;
ctx.createSerializer().growable(int8_t{}, [] (auto& s, auto& v) { });
ctx.createSerializer().growable(int8_t{}, [] (int8_t& v) { });
EXPECT_THAT(ctx.getBufferSize(), Eq(0));
ctx.bw->flush();
EXPECT_THAT(ctx.getBufferSize(), Gt(0));
@@ -60,7 +60,8 @@ TEST(SerializeGrowable, SessionDataConsistOfSessionsEndPosAnd2BytesSessionsDataO
constexpr size_t DATA_SIZE = 4;
int32_t data{};
ctx.createSerializer().growable(data, [](auto&s, auto& v) { s.value4b(v);});
auto ser = ctx.createSerializer();
ser.growable(data, [&ser](int32_t & v) { ser.value4b(v);});
ctx.createDeserializer();//to flush data and create buffer reader
EXPECT_THAT(ctx.getBufferSize(), Eq(3 + DATA_SIZE));

13
tests/serialization_objects.cpp
View File

@@ -49,26 +49,29 @@ struct Y {
struct Z { X x{}; Y y{}; };
SERIALIZE(Z)
template <typename S>
void serialize(S& s, Z& o)
{
s.object(o.x);
s.object(o.y);
}
SERIALIZE(X)
template <typename S>
void serialize(S& s, X& o)
{
s.template value<sizeof(o.x)>(o.x);
s.template text<1>(o.s, 1000);
}
SERIALIZE(Y)
template <typename S>
void serialize(S& s, Y& o)
{
auto writeInt = [](auto& s, auto& v) { s.template value<sizeof(v)>(v); };
auto writeInt = [&s]( int& v) { s.template value<sizeof(v)>(v); };
s.template text<1>(o.s, 10000);
s.template value<sizeof(o.y)>(o.y);
s.container(o.arr, writeInt);
s.container(o.carr, writeInt);
s.container(o.vx, 10000, [](auto& s, auto& v) { s.object(v); });
s.container(o.vx, 10000, [&s](X& v) { s.object(v); });
}

4
tests/serialization_size.cpp
View File

@@ -28,9 +28,9 @@ using testing::Eq;
bool SerializeDeserializeContainerSize(SerializationContext& ctx, const size_t size) {
std::vector<char> t1(size);
ctx.createSerializer().container(t1, size+1, [](auto , auto ){});
ctx.createSerializer().container(t1, size+1, []( char& ){});
t1.clear();
ctx.createDeserializer().container(t1, size+1, [](auto , auto ){});
ctx.createDeserializer().container(t1, size+1, []( char& ){});
return t1.size() == size;
}

24
tests/serialization_test_utils.h
View File

@@ -33,7 +33,7 @@
*/
struct MyStruct1 {
MyStruct1(int v1, int v2) : i1{v1}, i2{v2} {}
MyStruct1(int32_t v1, int32_t v2) : i1{v1}, i2{v2} {}
MyStruct1() : MyStruct1{0, 0} {}
@@ -47,12 +47,13 @@ struct MyStruct1 {
static constexpr size_t SIZE = sizeof(MyStruct1::i1) + sizeof(MyStruct1::i2);
};
SERIALIZE(MyStruct1) {
template <typename S>
void serialize(S& s, MyStruct1& o) {
s.template value<sizeof(o.i1)>(o.i1);
s.template value<sizeof(o.i2)>(o.i2);
}
enum class MyEnumClass {
enum class MyEnumClass:int32_t {
E1, E2, E3, E4, E5, E6
};
@@ -75,7 +76,8 @@ struct MyStruct2 {
static constexpr size_t SIZE = MyStruct1::SIZE + sizeof(MyStruct2::e1);
};
SERIALIZE(MyStruct2) {
template <typename S>
void serialize(S&s, MyStruct2& o) {
s.template value<sizeof(o.e1)>(o.e1);
s.object(o.s1);
}
@@ -87,9 +89,10 @@ public:
std::unique_ptr<bitsery::BufferWriter> bw;
std::unique_ptr<bitsery::BufferReader> br;
bitsery::Serializer<bitsery::BufferWriter> createSerializer() {
bw = std::make_unique<bitsery::BufferWriter>(buf);
return {*bw};
bitsery::Serializer createSerializer() {
//make_unique is not in c++11
bw = std::unique_ptr<bitsery::BufferWriter>(new bitsery::BufferWriter(buf));
return bitsery::Serializer{*bw};
};
size_t getBufferSize() const {
@@ -107,10 +110,11 @@ public:
return 4;
}
bitsery::Deserializer<bitsery::BufferReader> createDeserializer() {
bitsery::Deserializer createDeserializer() {
bw->flush();
br = std::make_unique<bitsery::BufferReader>(bw->getWrittenRange());
return {*br};
//make_unique is not in c++11
br = std::unique_ptr<bitsery::BufferReader>(new bitsery::BufferReader(bw->getWrittenRange()));
return bitsery::Deserializer{*br};
};
};

10
tests/serialization_text.cpp
View File

@@ -108,17 +108,11 @@ TEST(SerializeText, CArraySerializesTextLength) {
EXPECT_THAT(r1, ContainerEq(t1));
}
TEST(SerializeText, WhenCArrayNotNullterminatedThenMakeItNullterminated) {
TEST(SerializeText, WhenCArrayNotNullterminatedThenAssert) {
SerializationContext ctx;
char16_t t1[CARR_LENGTH]{u"some text"};
//make last character not nullterminated
t1[CARR_LENGTH-1] = 'x';
char16_t r1[CARR_LENGTH]{};
ctx.createSerializer().text<2>(t1);
ctx.createDeserializer().text<2>(r1);
EXPECT_THAT(ctx.getBufferSize(), Eq(ctx.containerSizeSerializedBytesCount(CARR_LENGTH) +
(CARR_LENGTH - 1) * 2));
EXPECT_THAT(r1[CARR_LENGTH-1], Eq(0));
EXPECT_DEATH(ctx.createSerializer().text<2>(t1), "");
}