From 397f13b7def2592f9a52c475937fc5f71e689de4 Mon Sep 17 00:00:00 2001 From: Boris Kolpackov Date: Fri, 8 Feb 2013 14:15:41 +0200 Subject: Document change-tracking containers --- NEWS | 19 +- doc/manual.xhtml | 566 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 554 insertions(+), 31 deletions(-) diff --git a/NEWS b/NEWS index 3eae712..f8ae6dc 100644 --- a/NEWS +++ b/NEWS @@ -25,6 +25,13 @@ Version 2.2.0 For more information, refer to the ODB manual "Type Mapping" sections for each database system. + * Support for change-tracking std::vector and QList container equivalents. + Change-tracking containers minimize the number of database operations + necessary to synchronize the container state with the database. For + more information, refer to Sections 5.4, "Change-Tracking Containers", + 5.4.1 "Change-Tracking vector", and 22.3.1, "Change-Tracking QList" + in the ODB manual. + * Support for automatically-derived SQL name (table, column, index, etc.) transformations. At the higher level, it is possible to assign prefixes and suffixes (--table-prefix, --{index,fkey,sequence}--suffix options) @@ -42,7 +49,7 @@ Version 2.2.0 manual. * Support for custom session implementations. For more information, refer - to Section 10.2, "Custom Session" in the ODB manual. + to Section 10.2, "Custom Sessions" in the ODB manual. * Support for early connection release. Now the database connection is released when commit()/rollback() is called rather than when the @@ -56,6 +63,16 @@ Version 2.2.0 information, refer to Section 19.2, "SQL Server Database Class" in the ODB manual. + * Support for "smart" containers. A smart container gets additional + functions which allow it to insert, update, and delete individual + elements in the database. Change-tracking containers are an example of + a smart container that utilizes this new functionality. Currently only + ordered smart containers are supported. Note also that with this + addition the names of the database functions provided by the ODB + compiler (see libodb/odb/container-traits.hxx) have changed. This will + only affect you if you have added ODB persistence support for custom + containers. + Version 2.1.0 * The ODB compiler is now capable of automatically discovering accessor and diff --git a/doc/manual.xhtml b/doc/manual.xhtml index ba4a75d..52928fe 100644 --- a/doc/manual.xhtml +++ b/doc/manual.xhtml @@ -356,7 +356,14 @@ for consistency. 5.1Ordered Containers 5.2Set and Multiset Containers 5.3Map and Multimap Containers - 5.4Using Custom Containers + + 5.4Change-Tracking Containers + + +
5.4.1Change-Tracking vector
+ + + 5.5Using Custom Containers @@ -438,7 +445,7 @@ for consistency. 10Session - +
10.1Object Cache
10.2Custom Session
10.2Custom Sessions
@@ -814,7 +821,13 @@ for consistency. 22.2Smart Pointers Library - 22.3Containers Library + + 22.3Containers Library + + +
22.3.1Change-Tracking QList
+ + 22.4Date Time Library @@ -1159,13 +1172,14 @@ for consistency.

1.3 Supported C++ Standards

-

ODB provides support for ISO/IEC C++ 1998 (C++98), ISO/IEC TR 19768 - C++ Library Extensions (C++ TR1), and ISO/IEC C++ 2011 (C++11). - While the majority of the examples in this manual use C++98, - support for the new functionality and library components introduced in - TR1 and C++11 are discussed throughout the document. The c++11 - example in the odb-examples package also shows ODB - support for various C++11 features.

+

ODB provides support for ISO/IEC C++ 1998/2003 (C++98/03), + ISO/IEC TR 19768 C++ Library Extensions (C++ TR1), and + ISO/IEC C++ 2011 (C++11). While the majority of the examples in + this manual use C++98/03, support for the new functionality and + library components introduced in TR1 and C++11 are discussed + throughout the document. The c++11 example in the + odb-examples package also shows ODB support for + various C++11 features.

@@ -5107,7 +5121,7 @@ for (age = 90; age > 40; age -= 10)

The cache_query() function caches the passed prepared query on the connection. The second overloaded version of cache_query() also takes a pointer to the - by-reference query parameters. In C++98 it should be + by-reference query parameters. In C++98/03 it should be std::auto_ptr while in C++11 std::auto_ptr or std::unique_ptr can be used. The cache_query() function assumes ownership of the @@ -5203,7 +5217,7 @@ for (unsigned short age (90); age > 40; age -= 10) query factory that will be called to prepare and cache a query during the call to lookup_query(). To register a factory we use the database::query_factory() - function. In C++98 it has the following signature:

+ function. In C++98/03 it has the following signature:

   void
@@ -5287,7 +5301,7 @@ db.query_factory (
   
5 Containers

 
   
The ODB runtime library provides built-in persistence support for all the
-     commonly used standard C++98 containers, namely,
+     commonly used standard C++98/03 containers, namely,
      std::vector, std::list, std::set,
      std::multiset, std::map, and
      std::multimap as well as C++11 std::array,
@@ -5297,9 +5311,14 @@ db.query_factory (
      Plus, ODB profile libraries, that are
      available for commonly used frameworks and libraries (such as Boost and
      Qt), provide persistence support for containers found in these frameworks
-     and libraries (Part III, "Profiles").  It is also easy
-     to persist custom container types as discussed later
-     in Section 5.4, "Using Custom Containers".

+     and libraries (Part III, "Profiles"). Both the
+     ODB runtime library and profile libraries also provide a number of
+     change-tracking container equivalents which can be used to minimize
+     the number of database operations necessary to synchronize the container
+     state with the database (Section 5.4, "Change-Tracking
+     Containers"). It is also easy to persist custom container types
+     as discussed later in Section 5.5, "Using Custom
+     Containers".

 
   
We don't need to do anything special to declare a member of a
      container type in a persistent class. For example:

@@ -5609,11 +5628,307 @@ private:
 };
-

5.4 Using Custom Containers

+

5.4 Change-Tracking Containers

+ +

When a persistent object containing one of the standard containers + is updated in the database, ODB has no knowledge of which elements + were inserted, erased, or modified. As a result, ODB has no choice + but to assume the whole container has changed and update the state + of every single element. This can result in a significant overhead + if a container contains a large number of elements and we only + changed a small subset of them.

+ +

To eliminate this overhead, ODB provides a notion of change-tracking + containers. A change-tracking container, besides containing + its elements, just like an ordinary container, also includes the + change state for each element. When it is time to update such a + container in the database, ODB can use this change information to + perform a minimum number of database operations necessary to + synchronize the container state with the database.

+ +

The current version of the ODB runtime library provides a change-tracking + equivalent of std::vector (Section 5.4.1, + "Change-Tracking vector") with support for other + standard container equivalents planned for future releases. ODB + profile libraries also provide change-tracking equivalents for some + containers found in the corresponding frameworks and libraries + (Part III, "Profiles").

+ +

A change-tracking container equivalent can normally be used as a drop-in + replacement for an ordinary container except for a few minor + interface differences (discussed in the corresponding sub-sections). + In particular, we don't need to do anything extra to effect + change tracking. ODB will automatically start, stop, and reset + change tracking when necessary. The following example illustrates + this point using odb::vector as a replacement for + std::vector.

+ + 
+#pragma db object
+class person
+{
+  ...
+
+  odb::vector<std::string> names;
+};
+
+person p; // No change tracking (not persistent).
+p.names.push_back ("John Doe");
+
+{
+  transaction t (db.begin ());
+  db.persist (p); // Start change tracking (persistent).
+  t.commit ();
+}
+
+p.names.push_back ("Johnny Doo");
+
+{
+  transaction t (db.begin ());
+  db.update (p); // One INSERT; reset change state.
+  t.commit ();
+}
+
+p.names.modify (0) = "Doe, John"; // Instead of operator[].
+p.names.pop_back ();
+
+{
+  transaction t (db.begin ());
+  db.update (p); // One UPDATE, one DELETE; reset change state.
+  t.commit ();
+}
+
+{
+  transaction t (db.begin ());
+  auto_ptr<person> p1 (db.load<person> (...)); // Start change tracking.
+  p1->names.insert (p1->names.begin (), "Joe Do");
+  db.update (*p1); // One UPDATE, one INSERT; reset change state.
+  t.commit ();
+}
+
+{
+  transaction t (db.begin ());
+  db.erase (p); // One DELETE; stop change tracking (not persistent).
+  t.commit ();
+}
+
+ +

One interesting aspect of change tracking is what happens when a + transaction that contains an update is later rolled back. In this + case, while the change-tracking container has reset the change + state (after update), actual changes were not committed to the + database. Change-tracking containers handle this case by + automatically registering a rollback callback and then, if it is + called, marking the container as "completely changed". In this + state, the container no longer tracks individual element changes + and, when updated, falls back to the complete state update, just + like an ordinary container. The following example illustrates + this point:

+ + 
+person p;
+p.names.push_back ("John Doe");
+
+{
+  transaction t (db.begin ());
+  db.persist (p); // Start change tracking (persistent).
+  t.commit ();
+}
+
+p.names.push_back ("Johnny Doo");
+
+for (;;)
+{
+  try
+  {
+    transaction t (db.begin ());
+
+    // First try: one INSERT.
+    // Next try: one DELETE, two INSERTs.
+    //
+    db.update (p); // Reset change state.
+
+    t.commit (); // If throws (rollback), mark as completely changed.
+    break;
+  }
+  catch (const odb::recoverable&)
+  {
+    continue;
+  }
+}
+
+ +

5.4.1 Change-Tracking vector

+ +

Class template odb::vector, defined in + <odb/vector.hxx>, is a change-tracking + equivalent for std::vector. It + is implemented in terms of std::vector and is + implicit-convertible to and implicit-constructible from + const std::vector&. In particular, this + means that we can use odb::vector instance + anywhere const std::vector& is + expected. In addition, odb::vector constant + iterator (const_iterator) is the same type as + that of std::vector.

+ +

odb::vector incurs 2-bit per element overhead + in order to store the change state. It cannot + be stored unordered in the database (Section + 12.4.18 "unordered") but can be used as an inverse + side of a relationship (6.2 "Bidirectional + Relationships"). In this case, no change tracking is performed + since no state for such a container is stored in the database.

+ +

The number of database operations required to update the state + of odb::vector corresponds well to the complexity + of std::vector functions. In particular, adding or + removing an element from the back of the vector (for example, + with push_back() and pop_back()), + requires only a single database statement execution. In contrast, + inserting or erasing an element somewhere in the middle of the + vector will require a database statement for every element that + follows it.

+ +

odb::vector replicates most of the std::vector + interface as defined in both C++98/03 and C++11 standards. However, + functions and operators that provide direct write access to + the elements had to be altered or disabled in order to support + change tracking. Additional functions used to interface with + std::vector and to control the change tracking state + were also added. The following listing summarizes the differences + between the odb::vector and std::vector + interfaces. Any std::vector function or operator + not mentioned in this listing has exactly the same signature + and semantics in odb::vector. Functions and + operators that were disabled are shown as commented out and + are followed by functions/operators that replace them.

+ + 
+namespace odb
+{
+  template <class T, class A = std::allocator<T> >
+  class vector
+  {
+    ...
+
+    // Element access.
+    //
+
+    //reference operator[] (size_type);
+      reference modify (size_type);
+
+    //reference at (size_type);
+      reference modify_at (size_type);
+
+    //reference front ();
+      reference modify_front ();
+
+    //reference back ();
+      reference modify_back ();
+
+    //T*        data () noexcept;
+      T*        modify_data () noexcept; // C++11 only.
+
+    // Iterators.
+    //
+    typedef typename std::vector<T, A>::const_iterator const_iterator;
+
+    class iterator
+    {
+      ...
+
+      // Element Access.
+      //
+
+      //reference       operator* () const;
+        const_reference operator* () const;
+        reference       modify () const;
+
+      //pointer       operator-> () const;
+        const_pointer operator-> () const;
+
+      //reference       operator[] (difference_type);
+        const_reference operator[] (difference_type);
+        reference       modify (difference_type) const;
+
+      // Interfacing with std::vector::iterator.
+      //
+      typename std::vector<T, A>::iterator base () const;
+    };
+
+    // Return std::vector iterators. The begin() functions mark
+    // all the elements as modified.
+    //
+    typename std::vector<T, A>::iterator         mbegin ();
+    typename std::vector<T, A>::iterator         mend ();
+    typename std::vector<T, A>::reverse_iterator mrbegin ();
+    typename std::vector<T, A>::reverse_iterator mrend ();
+
+    // Interfacing with std::vector.
+    //
+    vector (const std::vector<T, A>&);
+    vector (std::vector<T, A>&&); // C++11 only.
+
+    vector& operator= (const std::vector<T, A>&);
+    vector& operator= (std::vector<T, A>&&); // C++11 only.
+
+    operator const std::vector<T, A>& () const;
+    std::vector<T, A>& base ();
+    const std::vector<T, A>& base ();
+
+    // Change tracking.
+    //
+    bool _tracking () const;
+    void _start () const;
+    void _stop () const;
+    void _arm (transaction&) const;
+  };
+}
+
+ +

The following example highlights some of the differences between + the two interfaces. std::vector versions are commented + out.

+ + 
+#include <vector>
+#include <odb/vector.hxx>
+
+void f (const std::vector<int>&);
+
+odb::vector<int> v ({1, 2, 3});
+
+f (v); // Ok, implicit conversion.
+
+if (v[1] == 2) // Ok, const access.
+  //v[1]++;
+  v.modify (1)++;
+
+//v.back () = 4;
+v.modify_back () = 4;
+
+for (auto i (v.begin ()); i != v.end (); ++i)
+{
+  if (*i != 0) // Ok, const access.
+    //*i += 10;
+    i.modify () += 10;
+}
+
+std::sort (v.mbegin (), v.mend ());
+
+ +

Note also the subtle difference between copy/move construction + and copy/move assignment of odb::vector instances. + While copy/move constructor will copy/move both the elements as + well as their change state, in contrast, assignment is tracked + as any other change to the vector content.

+ +

5.5 Using Custom Containers

While the ODB runtime and profile libraries provide support for a wide range of containers, it is also easy to persist custom - container types.

+ container types or make a change-tracking version out of one.

To achieve this you will need to implement the container_traits class template specialization for @@ -9305,7 +9620,7 @@ struct employee_name object cache. In future versions it may provide additional functionality, such as delayed database operations and automatic object state change tracking. As discussed later in - Section 10.2, "Custom Session", it is also + Section 10.2, "Custom Sessions", it is also possible to provide a custom session implementation that provides these or other features.

@@ -9574,7 +9889,7 @@ unsigned long id2 (save (db, p2)); // p2 is cached in s as non-const. } -

10.2 Custom Session

+

10.2 Custom Sessions

ODB can use a custom session implementation instead of the default odb::session. There could be multiple @@ -15416,7 +15731,7 @@ namespace odb with short descriptions that are recognized by this constructor.

The last argument to all of the constructors is a pointer to the - connection factory. In C++98, it is std::auto_ptr while + connection factory. In C++98/03, it is std::auto_ptr while in C++11 std::unique_ptr is used instead. If we pass a non-NULL value, the database instance assumes ownership of the factory instance. The connection factory interface as well as @@ -16122,7 +16437,7 @@ auto_ptr<odb::database> db ( with short descriptions that are recognized by the third constructor.

The last argument to all of the constructors is a pointer to the - connection factory. In C++98, it is std::auto_ptr while + connection factory. In C++98/03, it is std::auto_ptr while in C++11 std::unique_ptr is used instead. If we pass a non-NULL value, the database instance assumes ownership of the factory instance. The connection factory interface as well as @@ -16992,7 +17307,7 @@ namespace odb with short descriptions that are recognized by this constructor.

The last argument to all of the constructors is a pointer to the - connection factory. In C++98, it is std::auto_ptr while + connection factory. In C++98/03, it is std::auto_ptr while in C++11 std::unique_ptr is used instead. If we pass a non-NULL value, the database instance assumes ownership of the factory instance. The connection factory interface as well as @@ -17781,7 +18096,7 @@ namespace odb ncharset arguments have no effect.

The last argument to all of the constructors is a pointer to the - connection factory. In C++98, it is std::auto_ptr while + connection factory. In C++98/03, it is std::auto_ptr while in C++11 std::unique_ptr is used instead. If we pass a non-NULL value, the database instance assumes ownership of the factory instance. The connection factory interface as well as @@ -18965,7 +19280,7 @@ odb::mssql::database dbA ("test", instance.

The last argument to all of the constructors is a pointer to the - connection factory. In C++98, it is std::auto_ptr while + connection factory. In C++98/03, it is std::auto_ptr while in C++11 std::unique_ptr is used instead. If we pass a non-NULL value, the database instance assumes ownership of the factory instance. The connection factory interface as well as @@ -20225,7 +20540,10 @@ class object

22 Qt Profile

The ODB profile implementation for Qt is provided by the - libodb-qt library and consists of multiple sub-profiles + libodb-qt library. Both Qt4 and Qt5 as well + as C++98/03 and C++11 are supported.

+ +

The Qt profile consists of multiple sub-profiles corresponding to the common type groups within Qt. Currently, only types from the QtCore module are supported. To enable all the available Qt sub-profiles, pass qt as the @@ -20248,7 +20566,7 @@ class object that can be thrown by the Qt sub-profiles are described in the following sections.

-

22.1 Basic Types

+

22.1 Basic Types Library

The basic sub-profile provides persistence support for basic types defined by Qt. To enable only this profile, pass @@ -20574,7 +20892,7 @@ class Person }; -

22.2 Smart Pointers

+

22.2 Smart Pointers Library

The smart-ptr sub-profile provides persistence support the Qt smart pointers. To enable only this profile, pass @@ -20645,7 +20963,7 @@ class Employee

22.3 Containers Library

-

The container sub-profile provides persistence support for +

The containers sub-profile provides persistence support for Qt containers. To enable only this profile, pass qt/containers to the --profile ODB compiler option.

@@ -20668,7 +20986,195 @@ class Person }; -

22.4 Date Time Types

+

The containers sub-profile also provide a change-tracking + equivalent for QList (Section 22.3.1, + "Change-Tracking QList") with support for other Qt + container equivalents planned for future releases. For general information + on change-tracking containers refer to Section 5.4, + "Change-Tracking Containers".

+ +

22.3.1 Change-Tracking QList

+ +

Class template QOdbList, defined in + <odb/qt/list.hxx>, is a change-tracking + equivalent for QList. It + is implemented in terms of QList and is + implicit-convertible to and implicit-constructible from + const QList&. In particular, this + means that we can use QOdbList instance + anywhere const QList& is + expected. In addition, QOdbList constant + iterator (const_iterator) is the same type as + that of QList.

+ +

QOdbList incurs 2-bit per element overhead + in order to store the change state. It cannot + be stored unordered in the database (Section + 12.4.18 "unordered") but can be used as an inverse + side of a relationship (6.2 "Bidirectional + Relationships"). In this case, no change tracking is performed + since no state for such a container is stored in the database.

+ +

The number of database operations required to update the state + of QOdbList corresponds well to the complexity + of QList functions, except for + prepend()/push_front(). In particular, adding + or removing an element from the back of the list (for example, + with append()/push_back() and + removeLast()/pop_back()), + requires only a single database statement execution. In contrast, + inserting or erasing an element at the beginning or in the middle + of the list will require a database statement for every element that + follows it.

+ +

QOdbList replicates most of the QList + interface as defined in both Qt4 and Qt5 and includes support for + C++11. However, functions and operators that provide direct write + access to the elements had to be altered or disabled in order to + support change tracking. Additional functions used to interface with + QList and to control the change tracking state + were also added. The following listing summarizes the differences + between the QOdbList and QList + interfaces. Any QList function or operator + not mentioned in this listing has exactly the same signature + and semantics in QOdbList. Functions and + operators that were disabled are shown as commented out and + are followed by functions/operators that replace them.

+ + 
+template <typename T>
+class QOdbList
+{
+  ...
+
+  // Element access.
+  //
+
+  //T& operator[] (int);
+    T& modify (int);
+
+  //T& first();
+    T& modifyFirst();
+
+  //T& last();
+    T& modifyLast();
+
+  //T& front();
+    T& modify_front();
+
+  //T& back();
+    T& modify_back();
+
+  // Iterators.
+  //
+  typedef typename QList<T>::const_iterator const_iterator;
+
+  class iterator
+  {
+    ...
+
+    // Element Access.
+    //
+
+    //reference       operator* () const;
+      const_reference operator* () const;
+      reference       modify () const;
+
+    //pointer       operator-> () const;
+      const_pointer operator-> () const;
+
+    //reference       operator[] (difference_type);
+      const_reference operator[] (difference_type);
+      reference       modify (difference_type) const;
+
+    // Interfacing with QList::iterator.
+    //
+    typename QList<T>::iterator base () const;
+  };
+
+  // Return QList iterators. The begin() functions mark all
+  // the elements as modified.
+  //
+  typename QList<T>::iterator mbegin ();
+  typename QList<T>::iterator modifyBegin ();
+  typename QList<T>::iterator mend ();
+  typename QList<T>::iterator modifyEnd ();
+
+  // Interfacing with QList.
+  //
+  QOdbList (const QList<T>&);
+  QOdbList (QList<T>&&); // C++11 only.
+
+  QOdbList& operator= (const QList<T>&);
+  QOdbList& operator= (QList<T>&&);
+
+  operator const QList<T>& () const;
+  QList<T>& base ();
+  const QList<T>& base () const;
+
+  // Change tracking.
+  //
+  bool _tracking () const;
+  void _start () const;
+  void _stop () const;
+  void _arm (transaction&) const;
+};
+
+ +

The following example highlights some of the differences between + the two interfaces. QList versions are commented + out.

+ + 
+#include <QtCore/QList>
+#include <odb/qt/list.hxx>
+
+void f (const QList<int>&);
+
+QOdbList<int> l ({1, 2, 3});
+
+f (l); // Ok, implicit conversion.
+
+if (l[1] == 2) // Ok, const access.
+  //l[1]++;
+  l.modify (1)++;
+
+//l.last () = 4;
+l.modifyLast () = 4;
+
+for (auto i (l.begin ()); i != l.end (); ++i)
+{
+  if (*i != 0) // Ok, const access.
+    //*i += 10;
+    i.modify () += 10;
+}
+
+qSort (l.modifyBegin (), l.modifyEnd ());
+
+ +

Note also the subtle difference between copy/move construction + and copy/move assignment of QOdbList instances. + While copy/move constructor will copy/move both the elements as + well as their change state, in contrast, assignment is tracked + as any other change to the vector content.

+ +

The QListIterator and QMutableListIterator + equivalents are also provided. These are QOdbListIterator + and QMutableOdbListIterator and are defined in + <odb/qt/list-iterator.hxx> and + <odb/qt/mutable-list-iterator.hxx>, respectively.

+ +

QOdbListIterator has exactly the same interface and + semantics as QListIterator. In fact, we can use + QListIterator to iterate over a QOdbList + instance.

+ +

QMutableOdbListIterator also has exactly the same + interface as QMutableListIterator. Note, however, + that any element that such an iterator passes over with the + call to next() is marked as modified.

+ +

22.4 Date Time Library

The date-time sub-profile provides persistence support for the Qt date-time types. To enable only this profile, pass -- cgit v1.1