[capnproto] MallocMessageBuilder mmap

[Sachin Kumar]

Hi,

I'm trying to use MallocMessageBuilder that's backed by a fixed size chunk
of mmap'ed memory. So, in pseudo-code:

char* buf = claim_mmap_memory(1024);
::capnp::MallocMessageBuilder message(buf);
... build message

The questions I have:

1) Am I correct in trying to do this with MallocMessageBuilder given that
the memory claimed is fixed size (1024 bytes in the above example). I can
generally assume that the message should *not* exceed 1024 -- perhaps I can
programmatically check the size used as the message is built and throw an
exception if it exceeds 1024?

2) Is it efficient to instance a malloc message builder like this in a
tight loop repeatedly to send many messages?

3) How do I convert the raw char* buffer into a kj::ArrayPtr<word> so that
I can pass into the constructor of MallocMessageBuilder?


Thanks,

Sachin

[Sachin Kumar]

Some updates from my own reading of other message on this group:

I figured out how to construct a kj::ArrayPtr<word> from a raw char*
pointer claimed from shared memory. Moreover, the raw buffer I get from my
mmap'ed file is 64-bit aligned. So this means there should be no issues
backing a MallocMessageBuilder with this memory, correct?

What still eludes me if this is an intended use case for MallocMessageBuilder,
particularly since I need to ensure that the MallocMessageBuilder does not
write more than the fixed size buffer I claimed, and that the
MallocMessageBuilder does not take ownership of the backed memory -- i.e.
does not try to deallocate it.

Any suggestions on this use case?

Post by Sachin Kumar
Hi,
I'm trying to use MallocMessageBuilder that's backed by a fixed size chunk
char* buf = claim_mmap_memory(1024);
::capnp::MallocMessageBuilder message(buf);
... build message
1) Am I correct in trying to do this with MallocMessageBuilder given that
the memory claimed is fixed size (1024 bytes in the above example). I can
generally assume that the message should *not* exceed 1024 -- perhaps I can
programmatically check the size used as the message is built and throw an
exception if it exceeds 1024?
2) Is it efficient to instance a malloc message builder like this in a
tight loop repeatedly to send many messages?
3) How do I convert the raw char* buffer into a kj::ArrayPtr<word> so
that I can pass into the constructor of MallocMessageBuilder?
Thanks,
Sachin

['Kenton Varda' via Cap'n Proto]

Hi Sachin,

MallocMessageBuilder is really intended to allocate space with malloc. If
you are using mmap, then you are not allocating using malloc, and
MallocMessageBuilder probably isn't the right way to go.

Instead, you should implement a custom subclass of MessageBuilder. Then,
when more space is needed, Cap'n Proto will call your custom
allocateSegment() implementation which can mmap more space (or throw an
exception, etc.).

Note also that by implementing a custom MessageBuilder subclass, you can
access the constructors of MessageBuilder that allow you to initialize it
with an existing message to be modified in-place. MallocMessageBuilder only
works for creating new messages from scratch. (However, note that there are
security concerns -- if you don't trust the message content, you should
always start a new message and copy the old message into it. See the
comments in the header file for more info.)

-Kenton

Post by Sachin Kumar
I figured out how to construct a kj::ArrayPtr<word> from a raw char*
pointer claimed from shared memory. Moreover, the raw buffer I get from my
mmap'ed file is 64-bit aligned. So this means there should be no issues
backing a MallocMessageBuilder with this memory, correct?
What still eludes me if this is an intended use case for MallocMessageBuilder,
particularly since I need to ensure that the MallocMessageBuilder does not
write more than the fixed size buffer I claimed, and that the
MallocMessageBuilder does not take ownership of the backed memory -- i.e.
does not try to deallocate it.
Any suggestions on this use case?

Post by Sachin Kumar
Hi,
I'm trying to use MallocMessageBuilder that's backed by a fixed size
char* buf = claim_mmap_memory(1024);
::capnp::MallocMessageBuilder message(buf);
... build message
1) Am I correct in trying to do this with MallocMessageBuilder given that
the memory claimed is fixed size (1024 bytes in the above example). I can
generally assume that the message should *not* exceed 1024 -- perhaps I can
programmatically check the size used as the message is built and throw an
exception if it exceeds 1024?
2) Is it efficient to instance a malloc message builder like this in a
tight loop repeatedly to send many messages?
3) How do I convert the raw char* buffer into a kj::ArrayPtr<word> so
that I can pass into the constructor of MallocMessageBuilder?
Thanks,
Sachin
--

You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/capnproto.