[capnproto] Dynamic reflection HowTo: Saving Schema as string and creating Dynamic Reader from string.

[Kerstin Keller]

Hi,

we have a communication middleware framework in place (quite similar to ROS
- with publish / subscribe patterns) and would like to integrate Capnproto
for message descriptions.
For Pub / Sub itself we have classes that can send and receive binary data
(to / from a std::string), so I'd like to build on that.
The idea is to have:

// Initialize a Publisher
Publisher<AddressBook> pub("address_book");
AddressBook::Builder addressBook = pub.getBuilder();

//manipulate the addressBook
auto people = addressBook.initPeople(2);
//...

// Send out the data
pub.Send();

So far, so good, I can send and receive data without problem.

The ecosystem, like ROS, also has recording possibilities. To be able to
interpret data on the fly and later on, I need the capnproto reflection
capabilities. So what I really need is to register the AddressBook (Binary)
Schema with the framework when a publisher is created.
But from the examples I really don't understand how this can be done. My
framework let's me store and retrieve reflection information as a
std::string. So I feel that I have to "serialize" the Schema information
into a string somehow, and rebuild it from a string. Since a schema also a
.capnp description it feels like this is something that can be done, but I
don't seem to grasp how it can be done.

So what I need is basically to implement these two functions:

template <typename T>
std::string getSchemaAsString<T>();

and

DynamicStruct::Reader getReader(const std::string& schema_string);

What I tried:

// This gives me the Schema. But is there a builder underneath?
auto schema = capnp::Schema::from<AddressBook>();

or do I need a SchemaLoader

capnp::SchemaLoader loader;
loader.loadCompiledTypeAndDependencies<AddressBook>();

And the other question, does it make sense to work with a `
MallocMessageBuilder` when I will later send out the data in a std::string
object, or does it make sense to Implement a `StdStringMessageBuilder`?

Any hints will be highly appreciated.

[Kerstin Keller]

Hi everyone,

I have been successful in saving the Schema to a std::string, and parsing
it back to a schema.
But it seems to be incorrect, since I can't parse the actual data later on.

I have created a standalone example to serialize both an AddressBook
message and an AddressBook Schema into a std::string, then recreate first
the schema from the std::string and use that schema to create a dynamic
message of the original AddressBook message. The sample compiles with both
VS 2015 and GCC 5.4 (of course, capnproto needs to be installed).

Using

capnp::SchemaLoader loader;
loader.loadCompiledTypeAndDependencies<AddressBook>();
auto schemas = loader.getAllLoaded();

creates an array of schemas, of which the first one is the schema for
`People`, not for `AddressBook`.
Using these to parse my data later on, produces a runtime exception.
If instead I am using only

auto schema = capnp::Schema::from<AddressBook>();

I can iterate over my data, but it will show the AddressBook as a list with
two entries, but not information about the list content is available.

Running the sample will produce the exception. I highly appreciate if
someone could take a quick look at the code.

['Kenton Varda' via Cap'n Proto]

Hi Kerstin,

Sorry for the slow reply.

It looks like your code is calling loader.getAllLoaded() to get the array
of schemas and then serializing them all, concatenated, into a single
string. So far so good. But on the receiving end, you are reading in this
string and parsing only the first schema from it. What you need to do is go
through the list and load all of the schemas, not just the first one. If
you use FlatArrayMessageReader to read each one, you can call its
`getEnd()` method to find out where the first message ended, which tells
you where the next message starts. Loop through all the schemas to load
them, then look up the schema for AddressBook (you'll probably have to look
it up by type ID rather than name) and use that to initialize the Dynamic
API.

Let me know if that helps.

-Kenton

Post by Kerstin Keller
Hi everyone,
I have been successful in saving the Schema to a std::string, and parsing
it back to a schema.
But it seems to be incorrect, since I can't parse the actual data later on.
I have created a standalone example to serialize both an AddressBook
message and an AddressBook Schema into a std::string, then recreate first
the schema from the std::string and use that schema to create a dynamic
message of the original AddressBook message. The sample compiles with both
VS 2015 and GCC 5.4 (of course, capnproto needs to be installed).
Using
capnp::SchemaLoader loader;
loader.loadCompiledTypeAndDependencies<AddressBook>();
auto schemas = loader.getAllLoaded();
creates an array of schemas, of which the first one is the schema for
`People`, not for `AddressBook`.
Using these to parse my data later on, produces a runtime exception.
If instead I am using only
auto schema = capnp::Schema::from<AddressBook>();
I can iterate over my data, but it will show the AddressBook as a list
with two entries, but not information about the list content is available.
Running the sample will produce the exception. I highly appreciate if
someone could take a quick look at the code.
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/capnproto.