release v5.2.1

BufferAdapter statically assert on underlying type size

bugfix in StdBitset

CHANGELOG.md

@@ -1,3 +1,21 @@
+# [5.2.1](https://github.com/fraillt/bitsery/compare/v5.2.0...v5.2.1) (2020-11-14)
+
+### Improvements
+* `Input/OutputBufferAdapter` now statically asserts that underlying type is 1byte in size.
+
+### Bug fixes
+* fixed serialization in `StdBitset` when it's size is less then `unsigned long long`.
+
+# [5.2.0](https://github.com/fraillt/bitsery/compare/v5.1.0...v5.2.0) (2020-11-09)
+
+### Features
+
+* new extension **StdBitset**.
+
+### Improvements
+* removed unused variable warnings in release build, where `max_size` variable during serialization is ignored.
+* removed unknown pragmas warnings for GCC/Clang (thanks to [Mmpuskas](https://github.com/Mmpuskas)).
+
 # [5.1.0](https://github.com/fraillt/bitsery/compare/v5.0.3...v5.1.0) (2020-06-08)
 
 ### Features
@@ -11,7 +29,7 @@
 * fixed some include paths, now you can basically to copy/paste bitsery include directory to your project without cmake support.
 
 ### Other notes
-* added tutorial of how to write your own extension ([here])(doc/tutorial/first_extension.md).
+* added tutorial of how to write your own extension ([here](doc/tutorial/first_extension.md)).
 * now gtest 1.10 is required if you want to build tests.
 
 # [5.0.3](https://github.com/fraillt/bitsery/compare/v5.0.2...v5.0.3) (2020-01-29)

CMakeLists.txt

@@ -1,7 +1,7 @@
 cmake_minimum_required(VERSION 3.1)
 project(bitsery
         LANGUAGES CXX
-        VERSION 5.1.0)
+        VERSION 5.2.1)
 
 #======== build options ===================================
 option(BITSERY_BUILD_EXAMPLES "Build examples" OFF)

CONTRIBUTING.md

@@ -6,7 +6,7 @@ However, to make sure the process of accepting patches goes smoothly, you should
 you contribute:
 
 1. Fork the repository.
-2. Create new branch based on the *master* branch (`git checkout -b your_branch master`). If your contribution is a bug fix, you should name your branch `bugfix/xxx`; for a feature, it should be `feature/xxx`. Otherwise, just use your good judgment. Consistent naming of branches is appreciated since it makes the output of `git branch` easier to understand with a single glance.
+2. Create new branch based on the *develop* branch (`git checkout -b your_branch develop`). If your contribution is a bug fix, you should name your branch `bugfix/xxx`; for a feature, it should be `feature/xxx`. Otherwise, just use your good judgment. Consistent naming of branches is appreciated since it makes the output of `git branch` easier to understand with a single glance.
 3. Do your modifications on that branch. Except for special cases, your contribution should include proper unit tests and documentation.
 4. Make sure your modifications did not break anything by building, running tests:
   ```shell
@@ -23,7 +23,7 @@ you contribute:
   ./show_coverage.sh build
   ```
 5. Commit your changes, and push to your fork (`git push origin your_branch`). Commit message should be one line short description. When applicable, please squash adjacent *wip* commits into a single *logical* commit.
-6. Open a pull request against Bitsery *master* branch. Currently ongoing development is on *master*. At some point an integration branch will be set-up, and pull-requests should target that, but for now its all against master. You may see feature branches come and go, too.
+6. Open a pull request against Bitsery *develop* branch.
 
 
 If you're working with visual studio, there is how to build and run all tests from command line

include/bitsery/adapter/buffer.h

@@ -39,6 +39,7 @@ namespace bitsery {
                       "Please define BufferAdapterTraits or include from <bitsery/traits/...>");
         static_assert(traits::ContainerTraits<typename std::remove_const<Buffer>::type>::isContiguous,
                       "BufferAdapter only works with contiguous containers");
+        static_assert(sizeof(TValue) == 1, "BufferAdapter underlying type must be 1byte.");
 
         InputBufferAdapter(TIterator beginIt, size_t size)
                 : _beginIt{beginIt},
@@ -196,6 +197,7 @@ namespace bitsery {
                       "Please define BufferAdapterTraits or include from <bitsery/traits/...>");
         static_assert(traits::ContainerTraits<Buffer>::isContiguous,
                       "BufferAdapter only works with contiguous containers");
+        static_assert(sizeof(TValue) == 1, "BufferAdapter underlying type must be 1byte.");
 
         OutputBufferAdapter(Buffer &buffer)
                 : _buffer{std::addressof(buffer)},

include/bitsery/bitsery.h

@@ -25,8 +25,8 @@
 #define BITSERY_BITSERY_H
 
 #define BITSERY_MAJOR_VERSION 5
-#define BITSERY_MINOR_VERSION 1
-#define BITSERY_PATCH_VERSION 0
+#define BITSERY_MINOR_VERSION 2
+#define BITSERY_PATCH_VERSION 1
 
 #define BITSERY_QUOTE_MACRO(name) #name
 #define BITSERY_BUILD_VERSION_STR(major,minor, patch) \

include/bitsery/details/adapter_common.h

@@ -146,12 +146,16 @@ namespace bitsery {
          */
         // add test data in separate struct, because some compilers only support constexpr functions with return-only body
         // suppress msvc warnings.
+#ifdef _MSC_VER
         #pragma warning( disable : 4310 )
+#endif
         struct EndiannessTestData {
             static constexpr uint32_t _sample4Bytes = 0x01020304;
             static constexpr uint8_t _sample1stByte = (const uint8_t &) _sample4Bytes;
         };
+#ifdef _MSC_VER
         #pragma warning( default : 4310 )
+#endif
 
         constexpr EndiannessType getSystemEndianness() {
             static_assert(EndiannessTestData::_sample1stByte == 0x04 || EndiannessTestData::_sample1stByte == 0x01,

include/bitsery/ext/std_bitset.h

@@ -0,0 +1,148 @@
+//MIT License
+//
+//Copyright (c) 2020 Mindaugas Vinkelis
+//
+//Permission is hereby granted, free of charge, to any person obtaining a copy
+//of this software and associated documentation files (the "Software"), to deal
+//in the Software without restriction, including without limitation the rights
+//to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+//copies of the Software, and to permit persons to whom the Software is
+//furnished to do so, subject to the following conditions:
+//
+//The above copyright notice and this permission notice shall be included in all
+//copies or substantial portions of the Software.
+//
+//THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+//IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+//FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+//AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+//LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+//OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+//SOFTWARE.
+
+#ifndef BITSERY_EXT_STD_BITSET_H
+#define BITSERY_EXT_STD_BITSET_H
+
+#include "../traits/core/traits.h"
+#include <bitset>
+
+namespace bitsery {
+    namespace ext {
+
+        class StdBitset {
+        public:
+
+            template<typename Ser, typename Fnc, size_t N>
+            void serialize(Ser &ser, const std::bitset<N> &obj, Fnc &&) const {
+                constexpr size_t BYTES = N / 8;
+                constexpr size_t LEFTOVER = N % 8;
+                if (BYTES > sizeof(unsigned long long)) {
+                    for(size_t i = 0u; i < BYTES; ++i) {
+                        size_t offset = i * 8;
+                        auto data = obj[offset + 0] +
+                                (obj[offset + 1] << 1) +
+                                (obj[offset + 2] << 2) +
+                                (obj[offset + 3] << 3) +
+                                (obj[offset + 4] << 4) +
+                                (obj[offset + 5] << 5) +
+                                (obj[offset + 6] << 6) +
+                                (obj[offset + 7] << 7);
+                        ser.value1b(static_cast<uint8_t>(data));
+                    }
+
+                } else {
+                    // more performant way
+                    auto data = obj.to_ullong();
+                    for(size_t i = 0u; i < BYTES; ++i) {
+                        ser.value1b(static_cast<uint8_t>(data & 0xFF));
+                        data >>= 8;
+                    }
+                }
+                if (LEFTOVER > 0) {
+                    serializeLeftover(ser.adapter(), obj, N - LEFTOVER, N);
+                }
+            }
+
+            template<typename Des, typename Fnc, size_t N>
+            void deserialize(Des &des, std::bitset<N> &obj, Fnc &&) const {
+                constexpr size_t BYTES = N / 8;
+                constexpr size_t LEFTOVER = N % 8;
+                for(size_t i = 0u; i < BYTES; ++i) {
+                    size_t offset = i * 8;
+                    uint8_t data = 0;
+                    des.value1b(data);
+                    obj[offset + 0] = data & 0x01u;
+                    obj[offset + 1] = data & 0x02u;
+                    obj[offset + 2] = data & 0x04u;
+                    obj[offset + 3] = data & 0x08u;
+                    obj[offset + 4] = data & 0x10u;
+                    obj[offset + 5] = data & 0x20u;
+                    obj[offset + 6] = data & 0x40u;
+                    obj[offset + 7] = data & 0x80u;
+                }
+                if (LEFTOVER > 0) {
+                    deserializeLeftover(des.adapter(), obj, N - LEFTOVER, N);
+                }
+            }
+
+        private:
+            template<typename Writer, size_t N>
+            void serializeLeftover(Writer& w, const std::bitset<N> &obj, size_t from, size_t to) const {
+                serializeLeftoverImpl(w, obj, from, to, std::integral_constant<bool, Writer::BitPackingEnabled> {});
+            }
+
+            template<typename Writer, size_t N>
+            void serializeLeftoverImpl(Writer& w, const std::bitset<N> &obj, size_t from, size_t to, std::integral_constant<bool, false>) const {
+                auto data = 0;
+                for (auto i = from; i < to; ++i) {
+                    data += obj[i] << (i - from);
+                }
+                w.template writeBytes<1>(static_cast<uint8_t>(data));
+            }
+
+            template<typename Writer, size_t N>
+            void serializeLeftoverImpl(Writer& w, const std::bitset<N> &obj, size_t from, size_t to, std::integral_constant<bool, true>) const {
+                for (auto i = from; i < to; ++i) {
+                    w.writeBits(obj[i], 1);
+                }
+            }
+
+            template<typename Reader, size_t N>
+            void deserializeLeftover(Reader& r, std::bitset<N> &obj, size_t from, size_t to) const {
+                deserializeLeftoverImpl(r, obj, from, to, std::integral_constant<bool, Reader::BitPackingEnabled> {});
+            }
+
+            template<typename Reader, size_t N>
+            void deserializeLeftoverImpl(Reader& r, std::bitset<N> &obj, size_t from, size_t to, std::integral_constant<bool, false>) const {
+                uint8_t data = 0u;
+                r.template readBytes<1>(data);
+                for (auto i = from; i < to; ++i) {
+                    obj[i] = data & (1u << (i - from));
+                }
+            }
+
+            template<typename Reader, size_t N>
+            void deserializeLeftoverImpl(Reader& r, std::bitset<N> &obj, size_t from, size_t to, std::integral_constant<bool, true>) const {
+                for (auto i = from; i < to; ++i) {
+                    uint8_t res = 0u;
+                    r.readBits(res, 1);
+                    obj[i] = res;
+                }
+            }
+
+        };
+    }
+
+    namespace traits {
+        template<size_t N>
+        struct ExtensionTraits<ext::StdBitset, std::bitset<N>> {
+            using TValue = void;
+            static constexpr bool SupportValueOverload = false;
+            static constexpr bool SupportObjectOverload = true;
+            static constexpr bool SupportLambdaOverload = false;
+        };
+    }
+}
+
+
+#endif //BITSERY_EXT_STD_BITSET_H

include/bitsery/ext/utils/pointer_utils.h

@@ -376,12 +376,12 @@ namespace bitsery {
                                      std::true_type, OwnershipType<PointerOwnershipType::Owner>) const {
                     const auto& ctx = des.template context<TPolymorphicContext<RTTI>>();
                     ctx.deserialize(des, TPtrManager<T>::getPtr(obj),
-                                     [&obj, this, memResource](
+                                     [&obj, memResource](
                                          const std::shared_ptr<PolymorphicHandlerBase>& handler) {
                                          TPtrManager<T>::createPolymorphic(obj, memResource, handler);
                                          return TPtrManager<T>::getPtr(obj);
                                      },
-                                     [&obj, memResource, this](const std::shared_ptr<PolymorphicHandlerBase>& handler) {
+                                     [&obj, memResource](const std::shared_ptr<PolymorphicHandlerBase>& handler) {
                                          TPtrManager<T>::destroyPolymorphic(obj, memResource, handler);
                                      });
                     ptrInfo.processOwner(TPtrManager<T>::getPtr(obj));
@@ -414,7 +414,7 @@ namespace bitsery {
                                                  obj, memResource, handler);
                                              return TPtrManager<T>::getPtr(obj);
                                          },
-                                         [&obj, memResource, this](const std::shared_ptr<PolymorphicHandlerBase>& handler) {
+                                         [&obj, memResource](const std::shared_ptr<PolymorphicHandlerBase>& handler) {
                                              TPtrManager<T>::destroyPolymorphic(obj, memResource, handler);
                                          });
                         if (!ptrInfo.sharedState)

include/bitsery/serializer.h

@@ -289,6 +289,7 @@ namespace bitsery {
             static_assert(traits::ContainerTraits<T>::isResizable,
                           "use container(const T&, Fnc) overload without `maxSize` for static containers");
             auto size = traits::ContainerTraits<T>::size(obj);
+            (void)maxSize; // unused in release
             assert(size <= maxSize);
             details::writeSize(this->_adapter, size);
             procContainer(std::begin(obj), std::end(obj), std::forward<Fnc>(fnc));
@@ -302,6 +303,7 @@ namespace bitsery {
                           "use container(const T&) overload without `maxSize` for static containers");
             static_assert(VSIZE > 0, "");
             auto size = traits::ContainerTraits<T>::size(obj);
+            (void)maxSize; // unused in release
             assert(size <= maxSize);
             details::writeSize(this->_adapter, size);
 
@@ -315,6 +317,7 @@ namespace bitsery {
             static_assert(traits::ContainerTraits<T>::isResizable,
                           "use container(const T&) overload without `maxSize` for static containers");
             auto size = traits::ContainerTraits<T>::size(obj);
+            (void)maxSize; // unused in release
             assert(size <= maxSize);
             details::writeSize(this->_adapter, size);
             procContainer(std::begin(obj), std::end(obj));
@@ -453,6 +456,7 @@ namespace bitsery {
         template<size_t VSIZE, typename T>
         void procText(const T& str, size_t maxSize) {
             const size_t length = traits::TextTraits<T>::length(str);
+            (void)maxSize; // unused in release
             assert((length + (traits::TextTraits<T>::addNUL ? 1u : 0u)) <= maxSize);
             details::writeSize(this->_adapter, length);
             auto begin = std::begin(str);

tests/serialization_container.cpp

@@ -166,7 +166,6 @@ TYPED_TEST(SerializeContainerDynamicSizeCompositeTypes, DefaultSerializeFunction
 
 TYPED_TEST(SerializeContainerDynamicSizeCompositeTypes, CustomFunctionThatDoNothing) {
     SerializationContext ctx{};
-    using TValue = typename TestFixture::TValue;
 
     ctx.createSerializer().container(this->src, 1000, EmptyFtor{});
     ctx.createDeserializer().container(this->res, 1000, EmptyFtor{});

tests/serialization_ext_std_bitset.cpp

@@ -0,0 +1,124 @@
+//MIT License
+//
+//Copyright (c) 2020 Mindaugas Vinkelis
+//
+//Permission is hereby granted, free of charge, to any person obtaining a copy
+//of this software and associated documentation files (the "Software"), to deal
+//in the Software without restriction, including without limitation the rights
+//to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+//copies of the Software, and to permit persons to whom the Software is
+//furnished to do so, subject to the following conditions:
+//
+//The above copyright notice and this permission notice shall be included in all
+//copies or substantial portions of the Software.
+//
+//THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+//IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+//FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+//AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+//LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+//OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+//SOFTWARE.
+
+#include <bitsery/ext/std_bitset.h>
+#include <bitsery/ext/value_range.h>
+
+#include <gmock/gmock.h>
+#include "serialization_test_utils.h"
+
+using StdBitset = bitsery::ext::StdBitset;
+using ValueRange = bitsery::ext::ValueRange<int>;
+
+using testing::Eq;
+
+TEST(SerializeExtensionStdBitset, BitsetSmallerThanULongLong) {
+    SerializationContext ctx;
+
+    std::bitset<31> data;
+    data[2] = true;
+    data[8] = true;
+    data[15] = true;
+    data[25] = true;
+    data[30] = true;
+    std::bitset<31> res;
+
+    ctx.createSerializer().ext(data, StdBitset{});
+    ctx.createDeserializer().ext(res, StdBitset{});
+    EXPECT_THAT(res, Eq(data));
+}
+
+TEST(SerializeExtensionStdBitset, BitsetSmallerThanULongLong2) {
+    SerializationContext ctx;
+
+    std::bitset<9> data;
+    data.set();
+    std::bitset<9> res;
+
+    ctx.createSerializer().ext(data, StdBitset{});
+    ctx.createDeserializer().ext(res, StdBitset{});
+    EXPECT_THAT(res, Eq(data));
+}
+
+
+TEST(SerializeExtensionStdBitset, BitsetLargerThanULongLong) {
+    SerializationContext ctx;
+
+    std::bitset<200> data;
+    data[1] = true;
+    data[31] = true;
+    data[63] = true;
+    data[100] = true;
+    data[191] = true;
+    std::bitset<200> res;
+
+    ctx.createSerializer().ext(data, StdBitset{});
+    ctx.createDeserializer().ext(res, StdBitset{});
+    EXPECT_THAT(res, Eq(data));
+}
+
+TEST(SerializeExtensionStdBitset, BitsetSmallerThanULongLongBitPackingEnabled) {
+    SerializationContext ctx;
+
+    std::bitset<12> data;
+    int other_data = 1001;
+    data[2] = true;
+    data[9] = true;
+    std::bitset<12> res{};
+    int other_res{};
+
+    ctx.createSerializer().enableBitPacking([&data, &other_data](SerializationContext::TSerializerBPEnabled& sbp) {
+        sbp.ext(data, StdBitset{});
+        sbp.ext(other_data, ValueRange{1000,1015});
+    });
+    ctx.createDeserializer().enableBitPacking([&res, &other_res](SerializationContext::TDeserializerBPEnabled& dbp) {
+        dbp.ext(res, StdBitset{});
+        dbp.ext(other_res, ValueRange{1000,1015});
+    });
+    EXPECT_THAT(res, Eq(data));
+    EXPECT_THAT(other_res, Eq(other_data));
+    EXPECT_THAT(ctx.getBufferSize(), Eq(2));
+}
+
+TEST(SerializeExtensionStdBitset, BitsetLargerThanULongLongBitPackingEnabled) {
+    SerializationContext ctx;
+
+    std::bitset<204> data;
+    int other_data = 1001;
+    data[1] = true;
+    data[100] = true;
+    data[191] = true;
+    std::bitset<204> res{};
+    int other_res{};
+
+    ctx.createSerializer().enableBitPacking([&data, &other_data](SerializationContext::TSerializerBPEnabled& sbp) {
+        sbp.ext(data, StdBitset{});
+        sbp.ext(other_data, ValueRange{1000,1015});
+    });
+    ctx.createDeserializer().enableBitPacking([&res, &other_res](SerializationContext::TDeserializerBPEnabled& dbp) {
+        dbp.ext(res, StdBitset{});
+        dbp.ext(other_res, ValueRange{1000,1015});
+    });
+    EXPECT_THAT(res, Eq(data));
+    EXPECT_THAT(other_res, Eq(other_data));
+    EXPECT_THAT(ctx.getBufferSize(), Eq(26));
+}