polymorphism improvements and new CompactValue extension

CHANGELOG.md

@@ -1,3 +1,17 @@
+# [4.4.0](https://github.com/fraillt/bitsery/compare/v4.3.0...v4.4.0) (2019-01-08)
+
+### Features
+* new extensions **CompactValue** and **CompactValueAsObject**, stores integral values in less space if possible. This is useful when you're working with mostly small values, that in rare cases can be large.
+E.g. `int64_t money = 8000;` will only use 2 bytes, instead of 8. **CompactValueAsObject** allows to use `ext()` overload, without specifying size of underlying type and sets BUFFER_OVERFLOW error if value doesn't fit in underlying type during deserialization.
+
+### Improvements
+* improved **PolymorphicContext** for registering base class hierarchies from different translation units.
+Previously there was only one symbol for `PolymorphicBaseClass`, but now a primary template for this type lives in anonymous namespace, so each translation unit could get their own symbol.
+`registerBasesList` was modified, so that where invocation happens, it will bind to correct symbol for `PolymorphicBaseClass`.
+This introduced breaking change, for those who used this syntax (`registerBasesList<MySerializer, Shape>({})`) during registration.
+It is encouraged to define helper type, that could be used for registering hierarchy for serialization and deserialization [example](examples/smart_pointers_with_polymorphism.cpp).
+* **PolymorphicContext** also get optional method `registerSingleBaseBranch`, that allows manually register hierarchies, but it is not recommended as it is error-prone.
+
 # [4.3.0](https://github.com/fraillt/bitsery/compare/v4.2.1...v4.3.0) (2018-08-23)
 
 ### Features
@@ -132,7 +146,7 @@ Be careful when using deserializing untrusted data and make sure to enforce fund
 ### Features
 
 * refactored interface, now works with C++11 compiler.
-* new new extension **Growable**, that allows to have forward/backward compatability within this functions serialization flow. It only allows to append new data at the end of to existing flow without breaking old consumers.
+* new extension **Growable**, that allows to have forward/backward compatability within this functions serialization flow. It only allows to append new data at the end of to existing flow without breaking old consumers.
   * old consumer: correctly read old interfce and ignore new data.
   * new consumer: get defaults (zero values) for new fields, when reading old data.
 * added new extension for associative *map* containers **ContainerMap**.

CMakeLists.txt

@@ -1,7 +1,7 @@
 cmake_minimum_required(VERSION 3.1)
 project(bitsery
         LANGUAGES CXX
-        VERSION 4.3.0)
+        VERSION 4.4.0)
 
 #======== build options ===================================
 option(BITSERY_BUILD_EXAMPLES "Build examples" OFF)

doc/README.md

@@ -28,6 +28,8 @@ Core Serializer/Deserializer functions (alphabetical order):
 
 Serializer/Deserializer extensions via `ext` method (alphabetical order):
 * `BaseClass` (4.2.0)
+* `CompactValue` (4.4.0)
+* `CompactValueAsObject` (4.4.0)
 * `Entropy` (3.0.0)
 * `Growable` (3.0.0)
 * `PointerOwner` (4.1.0)

examples/smart_pointers_with_polymorphism.cpp

@@ -177,6 +177,11 @@ namespace bitsery {
     }
 }
 
+// convenient type that stores all our types, so that we could easily register and
+// also it automatically ensures, that classes is registered in the same order for serialization and deserialization
+using MyPolymorphicClassesForRegistering = bitsery::ext::PolymorphicClassesList<Shape>;
+
+
 //use bitsery namespace for convenience
 using namespace bitsery;
 
@@ -228,13 +233,13 @@ int main() {
     Buffer buffer{};
     size_t writtenSize{};
     {
-        TContext ctx{};
-        MySerializer ser{OutputAdapter{buffer}, &ctx};
         //STEP 2
         //bind serializer with base polymorphic types, it will go through all reachable classes that is defined in first step.
         //so you dont need to add Rectangle to reach for RoundedRectangle
-        std::get<1>(ctx).registerBasesList(ser, ext::PolymorphicClassesList<Shape>{});
+        TContext ctx{};
+        std::get<1>(ctx).registerBasesList<MySerializer>(MyPolymorphicClassesForRegistering{});
         //serialize our data
+        MySerializer ser{OutputAdapter{buffer}, &ctx};
         ser.object(data);
         auto &w = AdapterAccess::getWriter(ser);
         w.flush();
@@ -248,10 +253,9 @@ int main() {
     SomeShapes res{};
     {
         TContext ctx{};
-        MyDeserializer des{InputAdapter{buffer.begin(), writtenSize}, &ctx};
+        std::get<1>(ctx).registerBasesList<MyDeserializer>(MyPolymorphicClassesForRegistering{});
-        //same as in serialization
-        std::get<1>(ctx).registerBasesList(des, ext::PolymorphicClassesList<Shape>{});
         //serialize our data
+        MyDeserializer des{InputAdapter{buffer.begin(), writtenSize}, &ctx};
         des.object(res);
         auto &r = AdapterAccess::getReader(des);
         //check if everything went find

include/bitsery/bitsery.h

@@ -25,7 +25,7 @@
 #define BITSERY_BITSERY_H
 
 #define BITSERY_MAJOR_VERSION 4
-#define BITSERY_MINOR_VERSION 3
+#define BITSERY_MINOR_VERSION 4
 #define BITSERY_PATCH_VERSION 0
 
 #define BITSERY_QUOTE_MACRO(name) #name

include/bitsery/details/adapter_common.h

@@ -29,6 +29,7 @@
 #include <vector>
 #include <stack>
 #include <cstring>
+#include <climits>
 #include "adapter_utils.h"
 #include "not_defined_type.h"
 
@@ -40,7 +41,7 @@ namespace bitsery {
 
         template<typename T>
         struct BitsSize:public std::integral_constant<size_t, sizeof(T) * 8> {
-
+            static_assert(CHAR_BIT == 8, "only support systems with byte size of 8 bits");
         };
 
         //add swap functions to class, to avoid compilation warning about unused functions

include/bitsery/ext/compact_value.h

@@ -0,0 +1,176 @@
+//MIT License
+//
+//Copyright (c) 2018 Mindaugas Vinkelis
+//
+//Permission is hereby granted, free of charge, to any person obtaining a copy
+//of this software and associated documentation files (the "Software"), to deal
+//in the Software without restriction, including without limitation the rights
+//to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+//copies of the Software, and to permit persons to whom the Software is
+//furnished to do so, subject to the following conditions:
+//
+//The above copyright notice and this permission notice shall be included in all
+//copies or substantial portions of the Software.
+//
+//THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+//IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+//FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+//AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+//LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+//OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+//SOFTWARE.
+
+#ifndef BITSERY_EXT_COMPACT_VALUE_H
+#define BITSERY_EXT_COMPACT_VALUE_H
+
+#include "../details/serialization_common.h"
+#include "../details/adapter_common.h"
+#include <cassert>
+
+namespace bitsery {
+
+    namespace details {
+
+        template <bool CheckOverflow>
+        class CompactValueImpl {
+        public:
+
+            template<typename Ser, typename Writer, typename T, typename Fnc>
+            void serialize(Ser &s, Writer &writer, const T &v, Fnc &&) const {
+                static_assert(std::is_integral<T>::value || std::is_enum<T>::value, "");
+                using TValue = typename IntegralFromFundamental<T>::TValue;
+                serializeImpl(s, writer, reinterpret_cast<const TValue&>(v), std::integral_constant<bool, sizeof(T) != 1>{});
+            }
+
+            template<typename Des, typename Reader, typename T, typename Fnc>
+            void deserialize(Des &d, Reader &reader, T &v, Fnc &&) const {
+                static_assert(std::is_integral<T>::value || std::is_enum<T>::value, "");
+                using TValue = typename IntegralFromFundamental<T>::TValue;
+                deserializeImpl(d, reader, reinterpret_cast<TValue &>(v), std::integral_constant<bool, sizeof(T) != 1>{});
+            }
+
+        private:
+
+            // if value is 1byte size, just serialize/ deserialize whole value
+            template<typename Ser, typename Writer, typename T>
+            void serializeImpl(Ser &s, Writer &, const T &v, std::false_type) const {
+                s.value1b(v);
+            }
+
+            template<typename Des, typename Reader, typename T>
+            void deserializeImpl(Des &d, Reader &, T &v, std::false_type) const {
+                d.value1b(v);
+            }
+
+            // when value is bigger than 1byte size,
+            template<typename Ser, typename Writer, typename T>
+            void serializeImpl(Ser &, Writer &writer, const T &v, std::true_type) const {
+                auto val = zigZagEncode(v, std::is_signed<typename IntegralFromFundamental<T>::TValue>{});
+                writeBytes(writer, val);
+            }
+
+
+            template<typename Des, typename Reader, typename T>
+            void deserializeImpl(Des &, Reader &reader, T &v, std::true_type) const {
+                using TUnsigned = SameSizeUnsigned<T>;
+                TUnsigned res{};
+                readBytes(reader, res);
+                v = zigZagDecode<T>(res, std::is_signed<typename IntegralFromFundamental<T>::TValue>{});
+            }
+
+            // zigzag encode signed types
+            template<typename T>
+            const SameSizeUnsigned<T> &zigZagEncode(const T &v, std::false_type) const {
+                return v;
+            }
+
+            template<typename TResult, typename TUnsigned>
+            const TResult &zigZagDecode(const TUnsigned &v, std::false_type) const{
+                return v;
+            }
+
+            template<typename T>
+            SameSizeUnsigned<T> zigZagEncode(const T &v, std::true_type) const {
+                return (v << 1) ^ (v >> (BitsSize<T>::value - 1));
+            }
+
+            template<typename TResult, typename TUnsigned>
+            TResult zigZagDecode(TUnsigned v, std::true_type) const {
+                return (v >> 1) ^ -(v & 1);
+            }
+
+            // write/read bytes one by one
+            template<typename Writer, typename T>
+            void writeBytes(Writer &w, const T &v) const {
+                auto val = v;
+                while(val > 0x7Fu) {
+                    w.template writeBytes<1>(static_cast<uint8_t>(val | 0x80u));
+                    val >>=7u;
+                }
+                w.template writeBytes<1>(static_cast<uint8_t>(val));
+            }
+
+            template<typename Reader, typename T>
+            void readBytes(Reader &r, T &v) const {
+                constexpr auto TBITS = sizeof(T)*8;
+                uint8_t b1{0x80u};
+                auto i = 0u;
+                for (;i < TBITS && b1 > 0x7Fu; i +=7u) {
+                    r.template readBytes<1>(b1);
+                    v += static_cast<T>(b1 & 0x7Fu) << i;
+                }
+                checkReadOverflow<Reader, T>(r, i, b1, std::integral_constant<bool, CheckOverflow>{});
+            }
+            template <typename Reader, typename T>
+            void checkReadOverflow(Reader &r, unsigned shiftedBy, uint8_t remainder, std::true_type) const {
+                constexpr auto TBITS = sizeof(T)*8;
+                if (shiftedBy > TBITS && remainder >> (TBITS + 7 - shiftedBy)) {
+                    r.setError(bitsery::ReaderError::DataOverflow);
+                }
+            }
+
+            template <typename Reader, typename T>
+            void checkReadOverflow(Reader &, unsigned , uint8_t , std::false_type) const {
+
+            }
+
+        };
+
+    }
+
+    namespace ext {
+
+        // this type will use value overload, and do not check if type is sufficiently large during deserialization
+        class CompactValue: public details::CompactValueImpl<false> {};
+
+        // this type will enable object overload, and set DataOverflow if value doesn't fit in type, during deserialization
+        class CompactValueAsObject: public details::CompactValueImpl<true> {};
+
+    }
+
+    namespace traits {
+        template<typename T>
+        struct ExtensionTraits<ext::CompactValue, T> {
+            using TValue = T;
+            static constexpr bool SupportValueOverload = true;
+            // disable object overload, because we don't have implemented serialization function for fundamental types
+            static constexpr bool SupportObjectOverload = false;
+            static constexpr bool SupportLambdaOverload = false;
+        };
+
+        template<typename T>
+        struct ExtensionTraits<ext::CompactValueAsObject, T> {
+            // use dummy implemenations for value and object overload
+            using TValue = void;
+            // only enable object overload
+            static constexpr bool SupportValueOverload = false;
+            static constexpr bool SupportObjectOverload = true;
+            static constexpr bool SupportLambdaOverload = false;
+        };
+
+    }
+
+}
+
+
+#endif //BITSERY_EXT_COMPACT_VALUE_H

include/bitsery/ext/utils/pointer_utils.h

@@ -70,6 +70,24 @@ namespace bitsery {
                           isSharedProcessed{false} {};
                 PointerOwnershipType ownershipType;
                 bool isSharedProcessed;
+
+                void update(PointerOwnershipType ptrType) {
+                    //do nothing for observer
+                    if (ptrType == PointerOwnershipType::Observer)
+                        return;
+                    if (ownershipType == PointerOwnershipType::Observer) {
+                        //set ownership type
+                        ownershipType = ptrType;
+                        return;
+                    }
+                    //only shared ownership can get here multiple times
+                    assert(ptrType == PointerOwnershipType::SharedOwner || ptrType == PointerOwnershipType::SharedObserver);
+                    //check if need to update to SharedOwner
+                    if (ptrType == PointerOwnershipType::SharedOwner)
+                        ownershipType = ptrType;
+                    //mark that object already processed, so we do not serialize/deserialize duplicate objects
+                    isSharedProcessed = true;
+                }
             };
 
             struct PLCInfoSerializer: PLCInfo {
@@ -111,24 +129,6 @@ namespace bitsery {
                 std::unique_ptr<PointerSharedStateBase> sharedState{};
             };
 
-            void updatePLCInfo(PLCInfo &ptrInfo, PointerOwnershipType ptrType) {
-                //do nothing for observer
-                if (ptrType == PointerOwnershipType::Observer)
-                    return;
-                if (ptrInfo.ownershipType == PointerOwnershipType::Observer) {
-                    //set ownership type
-                    ptrInfo.ownershipType = ptrType;
-                    return;
-                }
-                //only shared ownership can get here multiple times
-                assert(ptrType == PointerOwnershipType::SharedOwner || ptrType == PointerOwnershipType::SharedObserver);
-                //check if need to update to SharedOwner
-                if (ptrType == PointerOwnershipType::SharedOwner)
-                    ptrInfo.ownershipType = ptrType;
-                //mark that object already processed, so we do not serialize/deserialize duplicate objects
-                ptrInfo.isSharedProcessed = true;
-            }
-
             class PointerLinkingContextSerialization {
             public:
                 explicit PointerLinkingContextSerialization()
@@ -152,7 +152,7 @@ namespace bitsery {
                         ++_currId;
                         return ptrInfo;
                     }
-                    updatePLCInfo(ptrInfo, ptrType);
+                    ptrInfo.update(ptrType);
                     return ptrInfo;
                 }
 
@@ -191,7 +191,7 @@ namespace bitsery {
                     auto res = _idMap.emplace(id, PLCInfoDeserializer{nullptr, ptrType});
                     auto &ptrInfo = res.first->second;
                     if (!res.second)
-                        updatePLCInfo(ptrInfo, ptrType);
+                        ptrInfo.update(ptrType);
                     return ptrInfo;
                 }
 

include/bitsery/ext/utils/polymorphism_utils.h

@@ -44,10 +44,14 @@ namespace bitsery {
 // although you can add all derivates to same base like this:
 // template <> PolymorphicBaseClass<Animal>:PolymorphicDerivedClasses<Dog, Cat, Bulldog, GoldenRetriever>{};
 // it will not work when you try to serialize Dog*, because it will not find Bulldog and GoldenRetriever
-        template<typename TBase>
+        namespace {
-        struct PolymorphicBaseClass {
+            // this class must be in anonymous namespace, so that it would generate different symbols when defining different hierarchies in different translation units
-            using Childs = PolymorphicClassesList<>;
+            // https://github.com/fraillt/bitsery/issues/9
-        };
+            template<typename TBase>
+            struct PolymorphicBaseClass {
+                using Childs = PolymorphicClassesList<>;
+            };
+        }
 
 //derive from this class when specifying childs for your base class, atleast one child must exists, hence T1
 //e.g.
@@ -108,23 +112,23 @@ namespace bitsery {
                 }
             };
 
-            template<typename TSerializer, typename TBase, typename TDerived>
+            template<typename TSerializer, template<typename> class THierarchy, typename TBase, typename TDerived>
             void add() {
                 addToMap<TSerializer, TBase, TDerived>(std::is_abstract<TDerived>{});
-                addChilds<TSerializer, TBase, TDerived>(typename PolymorphicBaseClass<TDerived>::Childs{});
+                addChilds<TSerializer, THierarchy, TBase, TDerived>(typename THierarchy<TDerived>::Childs{});
             }
 
-            template<typename TSerializer, typename TBase, typename TDerived, typename T1, typename ... Tn>
+            template<typename TSerializer, template<typename> class THierarchy, typename TBase, typename TDerived, typename T1, typename ... Tn>
             void addChilds(PolymorphicClassesList<T1, Tn...>) {
                 static_assert(std::is_base_of<TDerived, T1>::value,
                               "PolymorphicBaseClass<TBase> must derive a list of derived classes from TBase.");
-                add<TSerializer, TBase, T1>();
+                add<TSerializer, THierarchy, TBase, T1>();
-                addChilds<TSerializer, TBase, TDerived>(PolymorphicClassesList<Tn...>{});
+                addChilds<TSerializer, THierarchy, TBase, TDerived>(PolymorphicClassesList<Tn...>{});
                 //iterate through derived class hierarchy as well
-                add<TSerializer, T1, T1>();
+                add<TSerializer, THierarchy, T1, T1>();
             }
 
-            template<typename TSerializer, typename TBase, typename TDerived>
+            template<typename TSerializer, template<typename> class THierarchy, typename TBase, typename TDerived>
             void addChilds(PolymorphicClassesList<>) {
             }
 
@@ -154,16 +158,39 @@ namespace bitsery {
                 _baseToDerivedArray.clear();
             }
 
-            template<typename TSerializer, typename T1, typename ...Tn>
+            template<typename TSerializer, template<typename> class THierarchy = PolymorphicBaseClass, typename T1, typename ...Tn>
-            void registerBasesList(const TSerializer &s, PolymorphicClassesList<T1, Tn...>) {
+            [[deprecated("de/serializer instance is not required")]] void registerBasesList(const TSerializer &s, PolymorphicClassesList<T1, Tn...>) {
-                add<TSerializer, T1, T1>();
+                add<TSerializer, THierarchy, T1, T1>();
-                registerBasesList<TSerializer>(s, PolymorphicClassesList<Tn...>{});
+                registerBasesList<TSerializer, THierarchy>(s, PolymorphicClassesList<Tn...>{});
             }
 
-            template<typename TSerializer>
+            template<typename TSerializer, template<typename> class THierarchy>
-            void registerBasesList(const TSerializer &, PolymorphicClassesList<>) {
+            [[deprecated]] void registerBasesList(const TSerializer &, PolymorphicClassesList<>) {
             }
 
+            // THierarchy is the name of class, that defines hierarchy
+            // PolymorphicBaseClass is defined as default parameter, so that at instantiation time
+            // it will get unique symbol in translation unit for PolymorphicBaseClass (which is defined in anonymous namespace)
+            // https://github.com/fraillt/bitsery/issues/9
+            template<typename TSerializer, template<typename> class THierarchy = PolymorphicBaseClass, typename T1, typename ...Tn>
+            void registerBasesList(PolymorphicClassesList<T1, Tn...>) {
+                add<TSerializer, THierarchy, T1, T1>();
+                registerBasesList<TSerializer, THierarchy>(PolymorphicClassesList<Tn...>{});
+            }
+
+            template<typename TSerializer, template<typename> class THierarchy>
+            void registerBasesList(PolymorphicClassesList<>) {
+            }
+
+            // optional method, in case you want to construct base class hierarchy your self
+            template <typename TSerializer, typename TBase, typename TDerived>
+            void registerSingleBaseBranch() {
+                static_assert(std::is_base_of<TBase, TDerived>::value, "TDerived must be derived from TBase");
+                static_assert(!std::is_abstract<TDerived>::value, "TDerived cannot be abstract");
+                addToMap<TSerializer, TBase, TDerived>(std::false_type{});
+            };
+
+
             template<typename Serializer, typename Writer, typename TBase>
             void serialize(Serializer &ser, Writer &writer, TBase &obj) {
                 //get derived key

tests/CMakeLists.txt

@@ -44,7 +44,7 @@ foreach (TestFile ${TestSourceFiles})
     add_executable(${TestName} ${TestFile})
     target_link_libraries(${TestName} PRIVATE GTest::Main Bitsery::bitsery)
     if (CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang")
-        target_compile_options(${TestName} PRIVATE -Wextra -Wno-missing-braces -Wpedantic -Weffc++)
+        target_compile_options(${TestName} PRIVATE -Wextra -Wno-missing-braces -Wpedantic -Weffc++ -Wno-c++14-extensions)
     endif()
 
     add_test(NAME ${TestName} COMMAND $<TARGET_FILE:${TestName}>)

tests/serialization_ext_compact_value.cpp

@@ -0,0 +1,241 @@
+//MIT License
+//
+//Copyright (c) 2017 Mindaugas Vinkelis
+//
+//Permission is hereby granted, free of charge, to any person obtaining a copy
+//of this software and associated documentation files (the "Software"), to deal
+//in the Software without restriction, including without limitation the rights
+//to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+//copies of the Software, and to permit persons to whom the Software is
+//furnished to do so, subject to the following conditions:
+//
+//The above copyright notice and this permission notice shall be included in all
+//copies or substantial portions of the Software.
+//
+//THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+//IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+//FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+//AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+//LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+//OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+//SOFTWARE.
+
+
+#include <bitsery/ext/compact_value.h>
+#include <gmock/gmock.h>
+#include "serialization_test_utils.h"
+
+#include <bitsery/traits/array.h>
+#include <iostream>
+#include <bitset>
+#include <chrono>
+
+using testing::Eq;
+using bitsery::ext::CompactValue;
+using bitsery::ext::CompactValueAsObject;
+using bitsery::EndiannessType;
+
+// helper function, that gets value filled with specified number of bits
+template <typename TValue>
+TValue getValue(bool isPositive, size_t significantBits) {
+    TValue v = isPositive ? 0 : -1;
+    if (significantBits == 0)
+        return v;
+
+    using TUnsigned = typename std::make_unsigned<TValue>::type;
+    TUnsigned mask = {};
+    mask = ~mask; // invert shiftByBits
+    auto shiftBy = bitsery::details::BitsSize<TValue>::value - significantBits;
+    mask >>= shiftBy;
+    //cast to unsigned when applying mask
+    return (TUnsigned)v ^ mask;
+}
+
+// helper function, that serialize and return deserialized value
+template <typename TSerContext, typename TValue>
+std::pair<TValue, size_t> serializeAndGetDeserialized(TValue data) {
+    TSerContext ctx;
+    TValue res{};
+    ctx.createSerializer().template ext<sizeof(TValue)>(data, CompactValue{});
+    ctx.createDeserializer().template ext<sizeof(TValue)>(res, CompactValue{});
+    return {res, ctx.getBufferSize()};
+}
+
+struct LittleEndianConfig: public bitsery::DefaultConfig {
+    static constexpr EndiannessType NetworkEndianness = EndiannessType::LittleEndian;
+};
+
+struct BigEndianConfig: public bitsery::DefaultConfig {
+    static constexpr EndiannessType NetworkEndianness = EndiannessType::BigEndian;
+};
+
+template <typename TValue, bool isPositiveNr, typename TConfig>
+struct TC {
+    static_assert(isPositiveNr || std::is_signed<TValue>::value, "");
+
+    using Value = TValue;
+    using Config = TConfig;
+    bool isPositive = isPositiveNr;
+};
+
+template<typename T>
+class SerializeExtensionCompactValueCorrectness : public testing::Test {
+public:
+    using TestCase = T;
+};
+
+
+using AllValueSizesTestCases = ::testing::Types<
+        TC<uint8_t, true, LittleEndianConfig>,
+        TC<uint16_t, true, LittleEndianConfig>,
+        TC<uint32_t, true, LittleEndianConfig>,
+        TC<uint64_t, true, LittleEndianConfig>,
+        TC<int8_t, true, LittleEndianConfig>,
+        TC<int16_t, true, LittleEndianConfig>,
+        TC<int32_t, true, LittleEndianConfig>,
+        TC<int64_t, true, LittleEndianConfig>,
+        TC<int8_t, false, LittleEndianConfig>,
+        TC<int16_t, false, LittleEndianConfig>,
+        TC<int32_t, false, LittleEndianConfig>,
+        TC<int64_t, false, LittleEndianConfig>,
+        TC<uint8_t, true, BigEndianConfig>,
+        TC<uint16_t, true, BigEndianConfig>,
+        TC<uint32_t, true, BigEndianConfig>,
+        TC<uint64_t, true, BigEndianConfig>,
+        TC<int8_t, true, BigEndianConfig>,
+        TC<int16_t, true, BigEndianConfig>,
+        TC<int32_t, true, BigEndianConfig>,
+        TC<int64_t, true, BigEndianConfig>,
+        TC<int8_t, false, BigEndianConfig>,
+        TC<int16_t, false, BigEndianConfig>,
+        TC<int32_t, false, BigEndianConfig>,
+        TC<int64_t, false, BigEndianConfig>
+>;
+
+TYPED_TEST_CASE(SerializeExtensionCompactValueCorrectness, AllValueSizesTestCases);
+
+TYPED_TEST(SerializeExtensionCompactValueCorrectness, TestDifferentSizeValues) {
+    using TCase = typename TestFixture::TestCase;
+    using TValue = typename TCase::Value;
+    TCase tc{};
+
+    for (auto i = 0u; i < bitsery::details::BitsSize<TValue>::value + 1; ++i) {
+        auto data = getValue<TValue>(tc.isPositive, i);
+        auto res = serializeAndGetDeserialized<BasicSerializationContext<typename TCase::Config, void>>(data);
+        EXPECT_THAT(res.first, Eq(data));
+    }
+}
+
+
+// this stucture will contain test data and result, as type paramters
+template <typename TValue, bool isPositiveNr, size_t significantBits, size_t resultBytes>
+struct SizeTC {
+    static_assert(isPositiveNr || std::is_signed<TValue>::value, "");
+    static_assert(bitsery::details::BitsSize<TValue>::value >= significantBits, "");
+
+    using Value = TValue;
+    bool isPositive = isPositiveNr;
+    size_t fillBits = significantBits;
+    size_t bytesCount = resultBytes;
+};
+
+template<typename T>
+class SerializeExtensionCompactValueRequiredBytes : public testing::Test {
+public:
+    using TestCase = T;
+};
+
+using RequiredBytesTestCases = ::testing::Types<
+        //1 byte always writes to 1 byte
+        SizeTC<uint8_t, true, 0,1>,
+        SizeTC<uint8_t, true, 8,1>,
+        SizeTC<int8_t, false, 0,1>,
+        SizeTC<int8_t, true, 8,1>,
+
+        //2 byte, +1 byte after 15 significant bits
+        SizeTC<uint16_t, true, 7,1>,
+        SizeTC<uint16_t, true, 8,2>,
+        SizeTC<uint16_t, true, 14,2>,
+        SizeTC<uint16_t, true, 15,3>,
+        //2 byte, +1 byte after 15-1 significant bits (1 bit for sign)
+        SizeTC<int16_t, true, 6,1>,
+        SizeTC<int16_t, false, 7,2>,
+        SizeTC<int16_t, true, 13,2>,
+        SizeTC<int16_t, false, 14,3>,
+
+        //4 byte, +1 byte after 29 significant bits
+        SizeTC<uint32_t, true, 14,2>,
+        SizeTC<uint32_t, true, 21,3>,
+        SizeTC<uint32_t, true, 28,4>,
+        SizeTC<uint32_t, true, 29,5>,
+        SizeTC<uint32_t, true, 32,5>,
+        //4 byte
+        SizeTC<int32_t, true, 13,2>,
+        SizeTC<int32_t, false, 20,3>,
+        SizeTC<int32_t, true, 27,4>,
+        SizeTC<int32_t, false, 28,5>,
+        SizeTC<int32_t, true, 31,5>,
+
+        //8 byte, +1 byte after 57 significant bits, or +2 byte when all bits are significant
+        SizeTC<uint64_t, true, 28,4>,
+        SizeTC<uint64_t, true, 35,5>,
+        SizeTC<uint64_t, true, 42,6>,
+        SizeTC<uint64_t, true, 49,7>,
+        SizeTC<uint64_t, true, 56,8>,
+        SizeTC<uint64_t, true, 57,9>,
+        SizeTC<uint64_t, true, 63,9>,
+        SizeTC<uint64_t, true, 64,10>,
+        //8 byte,
+        SizeTC<int64_t, true, 27,4>,
+        SizeTC<int64_t, false, 34,5>,
+        SizeTC<int64_t, true, 41,6>,
+        SizeTC<int64_t, false, 48,7>,
+        SizeTC<int64_t, true, 55,8>,
+        SizeTC<int64_t, false, 56,9>,
+        SizeTC<int64_t, true, 62,9>,
+        SizeTC<int64_t, false, 63,10>
+>;
+
+TYPED_TEST_CASE(SerializeExtensionCompactValueRequiredBytes, RequiredBytesTestCases);
+
+TYPED_TEST(SerializeExtensionCompactValueRequiredBytes, Test) {
+    using TCase = typename TestFixture::TestCase;
+    using TValue = typename TCase::Value;
+    TCase tc{};
+    TValue data = getValue<TValue>(tc.isPositive, tc.fillBits);
+    auto res = serializeAndGetDeserialized<SerializationContext>(data);
+    EXPECT_THAT(res.first, Eq(data));
+    EXPECT_THAT(res.second, tc.bytesCount);
+}
+
+enum b1En: uint8_t {
+    A,B,C,D=54,E
+};
+
+enum class b8En: int64_t {
+    A=-874987489,B,C=0,D,E=489748978, F,G
+};
+
+TEST(SerializeExtensionCompactValueEnum, TestEnums) {
+    auto d1 = b1En::E;
+    auto d2 = b8En::B;
+    auto d3 = b8En::F;
+    EXPECT_THAT(serializeAndGetDeserialized<SerializationContext>(d1).first, Eq(d1));
+    EXPECT_THAT(serializeAndGetDeserialized<SerializationContext>(d2).first, Eq(d2));
+    EXPECT_THAT(serializeAndGetDeserialized<SerializationContext>(d3).first, Eq(d3));
+}
+
+TEST(SerializeExtensionCompactValueAsObjectDeserializeOverflow, TestEnums) {
+    SerializationContext ctx;
+    auto data = getValue<uint32_t >(true, 17);
+    uint16_t res{};
+    auto& ser = ctx.createSerializer();
+    ser.ext(data, CompactValueAsObject{});
+    auto& des = ctx.createDeserializer();
+    des.ext(res, CompactValueAsObject{});
+    auto& rd = bitsery::AdapterAccess::getReader(des);
+    EXPECT_THAT(data, ::testing::Ne(res));
+    EXPECT_THAT(rd.error(), Eq(bitsery::ReaderError::DataOverflow));
+}
+
+

tests/serialization_ext_pointer_polymorphic_types.cpp

@@ -152,7 +152,7 @@ public:
         auto &res = sctx.createSerializer(&plctx);
         std::get<2>(plctx).clear();
         //bind serializer with classes
-        std::get<2>(plctx).registerBasesList(res, bitsery::ext::PolymorphicClassesList<Base>{});
+        std::get<2>(plctx).registerBasesList<SerContext::TSerializer>(bitsery::ext::PolymorphicClassesList<Base>{});
         return res;
     }
 
@@ -160,7 +160,7 @@ public:
         auto &res = sctx.createDeserializer(&plctx);
         std::get<2>(plctx).clear();
         //bind deserializer with classes
-        std::get<2>(plctx).registerBasesList(res, bitsery::ext::PolymorphicClassesList<Base>{});
+        std::get<2>(plctx).registerBasesList<SerContext::TDeserializer>(bitsery::ext::PolymorphicClassesList<Base>{});
         return res;
     }
 
@@ -322,7 +322,7 @@ TEST_F(SerializeExtensionPointerPolymorphicTypes,
     auto &des = sctx.createDeserializer(&plctx);
     auto &pc = std::get<2>(plctx);
     pc.clear();
-    pc.registerBasesList(des, bitsery::ext::PolymorphicClassesList<BaseClone>{});
+    pc.registerBasesList<SerContext::TDeserializer>(bitsery::ext::PolymorphicClassesList<BaseClone>{});
     des.ext(baseRes, PointerOwner{});
     EXPECT_THAT(sctx.br->error(), Eq(bitsery::ReaderError::InvalidPointer));
 }

tests/serialization_ext_std_smart_ptr.cpp

@@ -160,7 +160,7 @@ public:
         auto &res = sctx.createSerializer(&plctx);
         std::get<2>(plctx).clear();
         //bind serializer with classes
-        std::get<2>(plctx).registerBasesList(res, bitsery::ext::PolymorphicClassesList<Base>{});
+        std::get<2>(plctx).template registerBasesList<SerContext::TSerializer>(bitsery::ext::PolymorphicClassesList<Base>{});
         return res;
     }
 
@@ -168,7 +168,7 @@ public:
         auto &res = sctx.createDeserializer(&plctx);
         std::get<2>(plctx).clear();
         //bind deserializer with classes
-        std::get<2>(plctx).registerBasesList(res, bitsery::ext::PolymorphicClassesList<Base>{});
+        std::get<2>(plctx).template registerBasesList<SerContext::TDeserializer>(bitsery::ext::PolymorphicClassesList<Base>{});
         return res;
     }
 
@@ -407,7 +407,7 @@ public:
         auto &res = sctx.createSerializer(&plctx);
         std::get<2>(plctx).clear();
         //bind serializer with classes
-        std::get<2>(plctx).registerBasesList(res, bitsery::ext::PolymorphicClassesList<Base>{});
+        std::get<2>(plctx).registerBasesList<SerContext::TSerializer>(bitsery::ext::PolymorphicClassesList<Base>{});
         return res;
     }
 
@@ -415,7 +415,7 @@ public:
         auto &res = sctx.createDeserializer(&plctx);
         std::get<2>(plctx).clear();
         //bind deserializer with classes
-        std::get<2>(plctx).registerBasesList(res, bitsery::ext::PolymorphicClassesList<Base>{});
+        std::get<2>(plctx).registerBasesList<SerContext::TDeserializer>(bitsery::ext::PolymorphicClassesList<Base>{});
         return res;
     }