[capnproto] Segfault during JSON encode() in v0.6.1

[Marc Sune]

Hi guys,

I am experiencing a _very_ strange segfault during JSON encoding of message:

```
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 27946]
0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash,
true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1, capnp::(anonymous
namespace)::TypeHash, true> const&) ()
(gdb) bt
#0 0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash,
true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1, capnp::(anonymous
namespace)::TypeHash, true> const&) ()
#1 0x0000000001e4b11a in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_h1() const ()
#2 0x0000000001e4a976 in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_hash_code(capnp::Type const&) const ()
#3 0x0000000001e4a401 in std::_Hashtable<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::allocator<std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>

, std::__detail::_Select1st, std::equal_to<capnp::Type>, capnp::(anonymous

namespace)::TypeHash, std::__detail::_Mod_range_hashing,
std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy,
std::__detail::_Hashtable_traits<true, false, true> >::find(capnp::Type
const&) const ()
#4 0x0000000001e483a3 in std::unordered_map<capnp::Type,
capnp::JsonCodec::HandlerBase*, capnp::(anonymous namespace)::TypeHash,
std::equal_to<capnp::Type>, std::allocator<std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*> > >::find(capnp::Type const&) const ()
#5 0x0000000001e443d5 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#6 0x0000000001e44a6f in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#7 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#8 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#9 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#10 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#11 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#12 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#13 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#14 0x0000000001e45618 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#15 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#16 0x0000000001e45618 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#17 0x0000000001e43fcc in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type) const ()
#18 0x00000000009e02c7 in capnp::JsonCodec::encode<Message::Builder const&>
(this=0x7ffff22e1210, value=...) at /home/marc/.../capnp/compat/json.h:216
```

That's about all that I can get, even though capnproto is compiled with
DEBUG.

The message trying to be encoded (sorry, I am not sure I can share the
entire set of schemas), is a series of simple objects, which in the
inner-most object contains a list that is initialized normally:

115 s.initIfaceType(1);

The funny part; not initializing it, doesn't make JsonCodec crash. But
initializing it, or initializing it + setting a value (valid one), produces
the crash always.

Valgrind etc... doesn't complain until that point.

I am trying to isolate the problem, to make it reproducible, but I am not
able yet.

Any ideas on this?

Thanks

[Marc Sune]

Post by Marc Sune
That's about all that I can get, even though capnproto is compiled with

DEBUG.

Or shall I say, it should :/

Post by Marc Sune
Hi guys,
```
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 27946]
0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash,
true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1, capnp::(anonymous
namespace)::TypeHash, true> const&) ()
(gdb) bt
#0 0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash,
true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1, capnp::(anonymous
namespace)::TypeHash, true> const&) ()
#1 0x0000000001e4b11a in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_h1() const ()
#2 0x0000000001e4a976 in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_hash_code(capnp::Type const&) const ()
#3 0x0000000001e4a401 in std::_Hashtable<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::allocator<std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>

, std::__detail::_Select1st, std::equal_to<capnp::Type>, capnp::(anonymous

namespace)::TypeHash, std::__detail::_Mod_range_hashing,
std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy,
std::__detail::_Hashtable_traits<true, false, true> >::find(capnp::Type
const&) const ()
#4 0x0000000001e483a3 in std::unordered_map<capnp::Type,
capnp::JsonCodec::HandlerBase*, capnp::(anonymous namespace)::TypeHash,
std::equal_to<capnp::Type>, std::allocator<std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*> > >::find(capnp::Type const&) const ()
#5 0x0000000001e443d5 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#6 0x0000000001e44a6f in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#7 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#8 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#9 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#10 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#11 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#12 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#13 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#14 0x0000000001e45618 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#15 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#16 0x0000000001e45618 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#17 0x0000000001e43fcc in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type) const ()
#18 0x00000000009e02c7 in capnp::JsonCodec::encode<Message::Builder
const&> (this=0x7ffff22e1210, value=...) at
/home/marc/.../capnp/compat/json.h:216
```
That's about all that I can get, even though capnproto is compiled with
DEBUG.
The message trying to be encoded (sorry, I am not sure I can share the
entire set of schemas), is a series of simple objects, which in the
115 s.initIfaceType(1);
The funny part; not initializing it, doesn't make JsonCodec crash. But
initializing it, or initializing it + setting a value (valid one), produces
the crash always.
Valgrind etc... doesn't complain until that point.
I am trying to isolate the problem, to make it reproducible, but I am not
able yet.
Any ideas on this?
Thanks

['Kenton Varda' via Cap'n Proto]

Hi Marc,

Do you have any custom type handlers registered via addTypeHandler()? Is it
possible that the handler class has gone out-of-scope (been destroyed) by
the time the encoder is executed?

-Kenton

Post by Marc Sune

Post by Marc Sune
That's about all that I can get, even though capnproto is compiled with

DEBUG.
Or shall I say, it should :/

Post by Marc Sune
Hi guys,
```
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 27946]
0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash, true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash, true> const&) ()
(gdb) bt
#0 0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash, true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash, true> const&) ()
#1 0x0000000001e4b11a in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_h1() const ()
#2 0x0000000001e4a976 in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_hash_code(capnp::Type const&) const ()
#3 0x0000000001e4a401 in std::_Hashtable<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::allocator<std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*> >, std::__detail::_Select1st,
std::equal_to<capnp::Type>, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true,
false, true> >::find(capnp::Type const&) const ()
#4 0x0000000001e483a3 in std::unordered_map<capnp::Type,
capnp::JsonCodec::HandlerBase*, capnp::(anonymous namespace)::TypeHash,
std::equal_to<capnp::Type>, std::allocator<std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*> > >::find(capnp::Type const&) const ()
#5 0x0000000001e443d5 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#6 0x0000000001e44a6f in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#7 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#8 0x0000000001e45377 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#9 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#10 0x0000000001e45377 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#11 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#12 0x0000000001e45377 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#13 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#14 0x0000000001e45618 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#15 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#16 0x0000000001e45618 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#17 0x0000000001e43fcc in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type) const ()
#18 0x00000000009e02c7 in capnp::JsonCodec::encode<Message::Builder
const&> (this=0x7ffff22e1210, value=...) at /home/marc/.../capnp/compat/js
on.h:216
```
That's about all that I can get, even though capnproto is compiled with
DEBUG.
The message trying to be encoded (sorry, I am not sure I can share the
entire set of schemas), is a series of simple objects, which in the
115 s.initIfaceType(1);
The funny part; not initializing it, doesn't make JsonCodec crash. But
initializing it, or initializing it + setting a value (valid one), produces
the crash always.
Valgrind etc... doesn't complain until that point.
I am trying to isolate the problem, to make it reproducible, but I am not
able yet.
Any ideas on this?
Thanks

--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/capnproto.

[Marc Sune]

Kenton,

Post by 'Kenton Varda' via Cap'n Proto
Hi Marc,
Do you have any custom type handlers registered via addTypeHandler()? Is
it possible that the handler class has gone out-of-scope (been destroyed)
by the time the encoder is executed?

No, I am not using custom handlers. I am just dumping the Builder like this
(simplified):

template<typename T>
void dump_msg(T& msg){
try{
capnp::JsonCodec enc;
enc.setPrettyPrint(PPRINT);
auto t = enc.encode(msg);
fprintf(stderr, "MSG: %s\n", t.cStr());
}catch(...){}
}

Where, in this case, msg is:

147 capnp::MallocMessageBuilder builder;

148 auto msg = builder.initRoot<Message>();

//Fill it

254 dump_msg(msg);

So the builder and encoder al valid, I believe in the entire dumping.
Moreover, valgrind would complain before the SEGFAULT if something would be
out of the stack, and the only thing I get is the direct SEGFAULT.

Any thoughts? I will keep trying to isolate the problem

marc

-Kenton

Post by 'Kenton Varda' via Cap'n Proto

Post by Marc Sune

Post by Marc Sune
That's about all that I can get, even though capnproto is compiled with

DEBUG.
Or shall I say, it should :/

Post by Marc Sune
Hi guys,
```
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 27946]
0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash,
true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1, capnp::(anonymous
namespace)::TypeHash, true> const&) ()
(gdb) bt
#0 0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash,
true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1, capnp::(anonymous
namespace)::TypeHash, true> const&) ()
#1 0x0000000001e4b11a in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_h1() const ()
#2 0x0000000001e4a976 in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_hash_code(capnp::Type const&) const ()
#3 0x0000000001e4a401 in std::_Hashtable<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::allocator<std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>

, std::__detail::_Select1st, std::equal_to<capnp::Type>, capnp::(anonymous

namespace)::TypeHash, std::__detail::_Mod_range_hashing,
std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy,
std::__detail::_Hashtable_traits<true, false, true> >::find(capnp::Type
const&) const ()
#4 0x0000000001e483a3 in std::unordered_map<capnp::Type,
capnp::JsonCodec::HandlerBase*, capnp::(anonymous namespace)::TypeHash,
std::equal_to<capnp::Type>, std::allocator<std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*> > >::find(capnp::Type const&) const ()
#5 0x0000000001e443d5 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#6 0x0000000001e44a6f in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#7 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#8 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#9 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#10 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#11 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#12 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#13 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#14 0x0000000001e45618 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#15 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#16 0x0000000001e45618 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#17 0x0000000001e43fcc in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type) const ()
#18 0x00000000009e02c7 in capnp::JsonCodec::encode<Message::Builder
const&> (this=0x7ffff22e1210, value=...) at
/home/marc/.../capnp/compat/json.h:216
```
That's about all that I can get, even though capnproto is compiled with
DEBUG.
The message trying to be encoded (sorry, I am not sure I can share the
entire set of schemas), is a series of simple objects, which in the
115 s.initIfaceType(1);
The funny part; not initializing it, doesn't make JsonCodec crash. But
initializing it, or initializing it + setting a value (valid one), produces
the crash always.
Valgrind etc... doesn't complain until that point.
I am trying to isolate the problem, to make it reproducible, but I am
not able yet.
Any ideas on this?
Thanks

--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/capnproto.

['Kenton Varda' via Cap'n Proto]

Sorry, I don't really have any ideas here. The stack trace is deep in STL
code for the type handler map, inside find(). If you've registered no type
handlers, that map should be empty. It's hard to imagine how find() on an
empty std::unordered_map could ever segfault...

-Kenton

Post by Marc Sune
Kenton,

Post by 'Kenton Varda' via Cap'n Proto
Hi Marc,
Do you have any custom type handlers registered via addTypeHandler()? Is
it possible that the handler class has gone out-of-scope (been destroyed)
by the time the encoder is executed?

No, I am not using custom handlers. I am just dumping the Builder like
template<typename T>
void dump_msg(T& msg){
try{
capnp::JsonCodec enc;
enc.setPrettyPrint(PPRINT);
auto t = enc.encode(msg);
fprintf(stderr, "MSG: %s\n", t.cStr());
}catch(...){}
}
147 capnp::MallocMessageBuilder builder;
148 auto msg = builder.initRoot<Message>();
//Fill it
254 dump_msg(msg);
So the builder and encoder al valid, I believe in the entire dumping.
Moreover, valgrind would complain before the SEGFAULT if something would be
out of the stack, and the only thing I get is the direct SEGFAULT.
Any thoughts? I will keep trying to isolate the problem
marc
-Kenton

Post by 'Kenton Varda' via Cap'n Proto

Post by Marc Sune
That's about all that I can get, even though capnproto is compiled

with DEBUG.
Or shall I say, it should :/

Post by Marc Sune
Hi guys,
```
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 27946]
0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash, true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash, true> const&) ()
(gdb) bt
#0 0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash, true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash, true> const&) ()
#1 0x0000000001e4b11a in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_h1() const ()
#2 0x0000000001e4a976 in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_hash_code(capnp::Type const&) const ()
#3 0x0000000001e4a401 in std::_Hashtable<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::allocator<std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*> >, std::__detail::_Select1st,
std::equal_to<capnp::Type>, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true,
false, true> >::find(capnp::Type const&) const ()
#4 0x0000000001e483a3 in std::unordered_map<capnp::Type,
capnp::JsonCodec::HandlerBase*, capnp::(anonymous
namespace)::TypeHash, std::equal_to<capnp::Type>,
std::allocator<std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*> > >::find(capnp::Type const&) const ()
#5 0x0000000001e443d5 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#6 0x0000000001e44a6f in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#7 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#8 0x0000000001e45377 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#9 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#10 0x0000000001e45377 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#11 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#12 0x0000000001e45377 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#13 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#14 0x0000000001e45618 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#15 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#16 0x0000000001e45618 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#17 0x0000000001e43fcc in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type) const ()
#18 0x00000000009e02c7 in capnp::JsonCodec::encode<Message::Builder
const&> (this=0x7ffff22e1210, value=...) at /home/marc/.../capnp/compat/js
on.h:216
```
That's about all that I can get, even though capnproto is compiled with
DEBUG.
The message trying to be encoded (sorry, I am not sure I can share the
entire set of schemas), is a series of simple objects, which in the
115 s.initIfaceType(1);
The funny part; not initializing it, doesn't make JsonCodec crash. But
initializing it, or initializing it + setting a value (valid one), produces
the crash always.
Valgrind etc... doesn't complain until that point.
I am trying to isolate the problem, to make it reproducible, but I am
not able yet.
Any ideas on this?
Thanks

--
You received this message because you are subscribed to the Google
Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/capnproto.

--

You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/capnproto.

[Marc Sune]

Thanks Kenton,

An update on this. I am leaning to think it is an stack overflow, although
I couldn't confirm 100%. The truth is that, calling the same code from the
main(), with the same exact objects and encoding routine, doesn't crash.

However, when the encode() method is called from a thread that is created
by an external library, I consistently get a SIGSEGV. I've tried the
typical ulimit / pthread_attr_setstacksize() without much of a success (not
sure why though).

Re-arranging the code so that the encode() is called with less stack (I cut
2 or 3 frames), I get past the point of libstdc++, but I consistenly crash
here:

(gdb) bt
#0 0x00000000009d4e46 in capnp::_::DirectWireValue<unsigned int>::get
*(this=0x0)* at /home/marc/target/rootfs/include/capnp/endian.h:80
#1 0x0000000001e64d37 in
capnp::_::WirePointer::target(capnp::_::SegmentReader*) const ()
#2 0x0000000001e6a83c in
capnp::_::WireHelpers::readStructPointer(capnp::_::SegmentReader*,
capnp::_::CapTableReader*, capnp::_::WirePointer const*, capnp::word const*
, int) ()
#3 0x0000000001e5e7e7 in capnp::_::PointerReader::getStruct(capnp::word
const*) const ()
#4 0x0000000001e7fe61 in capnp::_::PointerHelpers<capnp::schema::Node,
(capnp::Kind)3>::get(capnp::_::PointerReader, capnp::word const*) ()
#5 0x0000000001e7f6ec in capnp::ReaderFor_<capnp::schema::Node,
(kind<capnp::schema::Node>)()>::Type
capnp::AnyPointer::Reader::getAs<capnp::schema::Node>() const ()
#6 0x0000000001e7de31 in capnp::schema::Node::Reader
capnp::readMessageUnchecked<capnp::schema::Node>(capnp::word const*) ()
#7 0x0000000001e780db in capnp::Schema::getProto() const ()
#8 0x0000000001e7995f in capnp::EnumSchema::getEnumerants() const ()
#9 0x0000000001e8068f in capnp::DynamicEnum::getEnumerant() const ()
#10 0x0000000001e45c74 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()

Really odd... So I think it is not related to capnproto code

Thanks
marc

Post by 'Kenton Varda' via Cap'n Proto
Sorry, I don't really have any ideas here. The stack trace is deep in STL
code for the type handler map, inside find(). If you've registered no type
handlers, that map should be empty. It's hard to imagine how find() on an
empty std::unordered_map could ever segfault...
-Kenton

Post by Marc Sune
Kenton,

Post by 'Kenton Varda' via Cap'n Proto
Hi Marc,
Do you have any custom type handlers registered via addTypeHandler()? Is
it possible that the handler class has gone out-of-scope (been destroyed)
by the time the encoder is executed?

No, I am not using custom handlers. I am just dumping the Builder like
template<typename T>
void dump_msg(T& msg){
try{
capnp::JsonCodec enc;
enc.setPrettyPrint(PPRINT);
auto t = enc.encode(msg);
fprintf(stderr, "MSG: %s\n", t.cStr());
}catch(...){}
}
147 capnp::MallocMessageBuilder builder;
148 auto msg = builder.initRoot<Message>();
//Fill it
254 dump_msg(msg);
So the builder and encoder al valid, I believe in the entire dumping.
Moreover, valgrind would complain before the SEGFAULT if something would be
out of the stack, and the only thing I get is the direct SEGFAULT.
Any thoughts? I will keep trying to isolate the problem
marc
-Kenton

Post by 'Kenton Varda' via Cap'n Proto

Post by Marc Sune
That's about all that I can get, even though capnproto is compiled

with DEBUG.
Or shall I say, it should :/

Post by Marc Sune
Hi guys,
```
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 27946]
0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash,
true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1, capnp::(anonymous
namespace)::TypeHash, true> const&) ()
(gdb) bt
#0 0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash,
true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1, capnp::(anonymous
namespace)::TypeHash, true> const&) ()
#1 0x0000000001e4b11a in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_h1() const ()
#2 0x0000000001e4a976 in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_hash_code(capnp::Type const&) const ()
#3 0x0000000001e4a401 in std::_Hashtable<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::allocator<std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>

, std::__detail::_Select1st, std::equal_to<capnp::Type>, capnp::(anonymous

namespace)::TypeHash, std::__detail::_Mod_range_hashing,
std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy,
std::__detail::_Hashtable_traits<true, false, true> >::find(capnp::Type
const&) const ()
#4 0x0000000001e483a3 in std::unordered_map<capnp::Type,
capnp::JsonCodec::HandlerBase*, capnp::(anonymous namespace)::TypeHash,
std::equal_to<capnp::Type>, std::allocator<std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*> > >::find(capnp::Type const&) const ()
#5 0x0000000001e443d5 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#6 0x0000000001e44a6f in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#7 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#8 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#9 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#10 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#11 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#12 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#13 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#14 0x0000000001e45618 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#15 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#16 0x0000000001e45618 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#17 0x0000000001e43fcc in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type) const ()
#18 0x00000000009e02c7 in capnp::JsonCodec::encode<Message::Builder
const&> (this=0x7ffff22e1210, value=...) at
/home/marc/.../capnp/compat/json.h:216
```
That's about all that I can get, even though capnproto is compiled
with DEBUG.
The message trying to be encoded (sorry, I am not sure I can share the
entire set of schemas), is a series of simple objects, which in the
115 s.initIfaceType(1);
The funny part; not initializing it, doesn't make JsonCodec crash. But
initializing it, or initializing it + setting a value (valid one), produces
the crash always.
Valgrind etc... doesn't complain until that point.
I am trying to isolate the problem, to make it reproducible, but I am
not able yet.
Any ideas on this?
Thanks

--
You received this message because you are subscribed to the Google
Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/capnproto.

--

You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/capnproto.

[Marc Sune]

Kenton,

Is it remotely possible some of the encode() methods are consuming large
amounts of stack? I would need to to get deep into the code, but glancing
over the code I see some parameters that _seem_ to be passed by value, and
few autos which I am _not sure_ on whether they might perform a copy or not.

Marc

Post by Marc Sune
Thanks Kenton,
An update on this. I am leaning to think it is an stack overflow, although
I couldn't confirm 100%. The truth is that, calling the same code from the
main(), with the same exact objects and encoding routine, doesn't crash.
However, when the encode() method is called from a thread that is created
by an external library, I consistently get a SIGSEGV. I've tried the
typical ulimit / pthread_attr_setstacksize() without much of a success (not
sure why though).
Re-arranging the code so that the encode() is called with less stack (I
cut 2 or 3 frames), I get past the point of libstdc++, but I consistenly
(gdb) bt
#0 0x00000000009d4e46 in capnp::_::DirectWireValue<unsigned int>::get
*(this=0x0)* at /home/marc/target/rootfs/include/capnp/endian.h:80
#1 0x0000000001e64d37 in
capnp::_::WirePointer::target(capnp::_::SegmentReader*) const ()
#2 0x0000000001e6a83c in
capnp::_::WireHelpers::readStructPointer(capnp::_::SegmentReader*,
capnp::_::CapTableReader*, capnp::_::WirePointer const*, capnp::word const*
, int) ()
#3 0x0000000001e5e7e7 in capnp::_::PointerReader::getStruct(capnp::word
const*) const ()
#4 0x0000000001e7fe61 in capnp::_::PointerHelpers<capnp::schema::Node,
(capnp::Kind)3>::get(capnp::_::PointerReader, capnp::word const*) ()
#5 0x0000000001e7f6ec in capnp::ReaderFor_<capnp::schema::Node,
(kind<capnp::schema::Node>)()>::Type
capnp::AnyPointer::Reader::getAs<capnp::schema::Node>() const ()
#6 0x0000000001e7de31 in capnp::schema::Node::Reader
capnp::readMessageUnchecked<capnp::schema::Node>(capnp::word const*) ()
#7 0x0000000001e780db in capnp::Schema::getProto() const ()
#8 0x0000000001e7995f in capnp::EnumSchema::getEnumerants() const ()
#9 0x0000000001e8068f in capnp::DynamicEnum::getEnumerant() const ()
#10 0x0000000001e45c74 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
Really odd... So I think it is not related to capnproto code
Thanks
marc

Post by 'Kenton Varda' via Cap'n Proto
Sorry, I don't really have any ideas here. The stack trace is deep in STL
code for the type handler map, inside find(). If you've registered no type
handlers, that map should be empty. It's hard to imagine how find() on an
empty std::unordered_map could ever segfault...
-Kenton

Post by Marc Sune
Kenton,

Post by 'Kenton Varda' via Cap'n Proto
Hi Marc,
Do you have any custom type handlers registered via addTypeHandler()?
Is it possible that the handler class has gone out-of-scope (been
destroyed) by the time the encoder is executed?

No, I am not using custom handlers. I am just dumping the Builder like
template<typename T>
void dump_msg(T& msg){
try{
capnp::JsonCodec enc;
enc.setPrettyPrint(PPRINT);
auto t = enc.encode(msg);
fprintf(stderr, "MSG: %s\n", t.cStr());
}catch(...){}
}
147 capnp::MallocMessageBuilder builder;
148 auto msg = builder.initRoot<Message>();
//Fill it
254 dump_msg(msg);
So the builder and encoder al valid, I believe in the entire dumping.
Moreover, valgrind would complain before the SEGFAULT if something would be
out of the stack, and the only thing I get is the direct SEGFAULT.
Any thoughts? I will keep trying to isolate the problem
marc
-Kenton

Post by 'Kenton Varda' via Cap'n Proto

Post by Marc Sune
That's about all that I can get, even though capnproto is compiled

with DEBUG.
Or shall I say, it should :/

Post by Marc Sune
Hi guys,
```
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 27946]
0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash,
true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1, capnp::(anonymous
namespace)::TypeHash, true> const&) ()
(gdb) bt
#0 0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash,
true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1, capnp::(anonymous
namespace)::TypeHash, true> const&) ()
#1 0x0000000001e4b11a in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_h1() const ()
#2 0x0000000001e4a976 in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_hash_code(capnp::Type const&) const ()
#3 0x0000000001e4a401 in std::_Hashtable<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::allocator<std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>

, std::__detail::_Select1st, std::equal_to<capnp::Type>, capnp::(anonymous

namespace)::TypeHash, std::__detail::_Mod_range_hashing,
std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy,
std::__detail::_Hashtable_traits<true, false, true> >::find(capnp::Type
const&) const ()
#4 0x0000000001e483a3 in std::unordered_map<capnp::Type,
capnp::JsonCodec::HandlerBase*, capnp::(anonymous namespace)::TypeHash,
std::equal_to<capnp::Type>, std::allocator<std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*> > >::find(capnp::Type const&) const ()
#5 0x0000000001e443d5 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#6 0x0000000001e44a6f in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#7 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#8 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#9 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#10 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#11 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#12 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#13 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#14 0x0000000001e45618 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#15 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#16 0x0000000001e45618 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#17 0x0000000001e43fcc in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type) const ()
#18 0x00000000009e02c7 in capnp::JsonCodec::encode<Message::Builder
const&> (this=0x7ffff22e1210, value=...) at
/home/marc/.../capnp/compat/json.h:216
```
That's about all that I can get, even though capnproto is compiled
with DEBUG.
The message trying to be encoded (sorry, I am not sure I can share
the entire set of schemas), is a series of simple objects, which in the
115 s.initIfaceType(1);
The funny part; not initializing it, doesn't make JsonCodec crash.
But initializing it, or initializing it + setting a value (valid one),
produces the crash always.
Valgrind etc... doesn't complain until that point.
I am trying to isolate the problem, to make it reproducible, but I am
not able yet.
Any ideas on this?
Thanks

--
You received this message because you are subscribed to the Google
Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/capnproto.

--

You received this message because you are subscribed to the Google
Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/capnproto.

['Kenton Varda' via Cap'n Proto]

Hi Marc,

I would guess each encode() stack frame is on the order of 100 bytes.
Unless your stacks are *really* small, that doesn't seem like it could be
the problem.

It looks like the real problem has something to do with unexpectedly null
pointers appearing in the schema objects. These objects are declared as
static constants in the generated code. Here's an example:

const ::capnp::_::RawSchema s_e682ab4cf923a417 = {
0xe682ab4cf923a417, b_e682ab4cf923a417.words, 225, d_e682ab4cf923a417,
m_e682ab4cf923a417,
8, 14, i_e682ab4cf923a417, nullptr, nullptr, { &s_e682ab4cf923a417,
nullptr, nullptr, 0, 0, nullptr }
}

Your latest stack trace shows a case where, somehow, the second field of
this turned out null, which seems impossible.

Is it at all possible that your generated code was created with a different
version of Cap'n Proto compiler vs. the runtime library and/or headers you
are compiling against?

-Kenton

Post by Marc Sune
Kenton,
Is it remotely possible some of the encode() methods are consuming large
amounts of stack? I would need to to get deep into the code, but glancing
over the code I see some parameters that _seem_ to be passed by value, and
few autos which I am _not sure_ on whether they might perform a copy or not.
Marc

Post by Marc Sune
Thanks Kenton,
An update on this. I am leaning to think it is an stack overflow,
although I couldn't confirm 100%. The truth is that, calling the same code
from the main(), with the same exact objects and encoding routine, doesn't
crash.
However, when the encode() method is called from a thread that is created
by an external library, I consistently get a SIGSEGV. I've tried the
typical ulimit / pthread_attr_setstacksize() without much of a success (not
sure why though).
Re-arranging the code so that the encode() is called with less stack (I
cut 2 or 3 frames), I get past the point of libstdc++, but I consistenly
(gdb) bt
#0 0x00000000009d4e46 in capnp::_::DirectWireValue<unsigned int>::get
*(this=0x0)* at /home/marc/target/rootfs/include/capnp/endian.h:80
#1 0x0000000001e64d37 in capnp::_::WirePointer::target(capnp::_::SegmentReader*)
const ()
#2 0x0000000001e6a83c in capnp::_::WireHelpers::readStr
uctPointer(capnp::_::SegmentReader*, capnp::_::CapTableReader*,
capnp::_::WirePointer const*, capnp::word const*
, int) ()
#3 0x0000000001e5e7e7 in capnp::_::PointerReader::getStruct(capnp::word
const*) const ()
#4 0x0000000001e7fe61 in capnp::_::PointerHelpers<capnp::schema::Node,
(capnp::Kind)3>::get(capnp::_::PointerReader, capnp::word const*) ()
#5 0x0000000001e7f6ec in capnp::ReaderFor_<capnp::schema::Node,
(kind<capnp::schema::Node>)()>::Type capnp::AnyPointer::Reader::getAs<capnp::schema::Node>()
const ()
#6 0x0000000001e7de31 in capnp::schema::Node::Reader
capnp::readMessageUnchecked<capnp::schema::Node>(capnp::word const*) ()
#7 0x0000000001e780db in capnp::Schema::getProto() const ()
#8 0x0000000001e7995f in capnp::EnumSchema::getEnumerants() const ()
#9 0x0000000001e8068f in capnp::DynamicEnum::getEnumerant() const ()
#10 0x0000000001e45c74 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
Really odd... So I think it is not related to capnproto code
Thanks
marc

Post by 'Kenton Varda' via Cap'n Proto
Sorry, I don't really have any ideas here. The stack trace is deep in
STL code for the type handler map, inside find(). If you've registered no
type handlers, that map should be empty. It's hard to imagine how find() on
an empty std::unordered_map could ever segfault...
-Kenton

Post by Marc Sune
Kenton,

Post by 'Kenton Varda' via Cap'n Proto
Hi Marc,
Do you have any custom type handlers registered via addTypeHandler()?
Is it possible that the handler class has gone out-of-scope (been
destroyed) by the time the encoder is executed?

No, I am not using custom handlers. I am just dumping the Builder like
template<typename T>
void dump_msg(T& msg){
try{
capnp::JsonCodec enc;
enc.setPrettyPrint(PPRINT);
auto t = enc.encode(msg);
fprintf(stderr, "MSG: %s\n", t.cStr());
}catch(...){}
}
147 capnp::MallocMessageBuilder builder;
148 auto msg = builder.initRoot<Message>();
//Fill it
254 dump_msg(msg);
So the builder and encoder al valid, I believe in the entire dumping.
Moreover, valgrind would complain before the SEGFAULT if something would be
out of the stack, and the only thing I get is the direct SEGFAULT.
Any thoughts? I will keep trying to isolate the problem
marc
-Kenton

Post by 'Kenton Varda' via Cap'n Proto

Post by Marc Sune
That's about all that I can get, even though capnproto is compiled

with DEBUG.
Or shall I say, it should :/

Post by Marc Sune
Hi guys,
```
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 27946]
0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash, true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash, true> const&) ()
(gdb) bt
#0 0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash, true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash, true> const&) ()
#1 0x0000000001e4b11a in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_h1() const ()
#2 0x0000000001e4a976 in std::__detail::_Hash_code_base<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::__detail::_Select1st, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
true>::_M_hash_code(capnp::Type const&) const ()
#3 0x0000000001e4a401 in std::_Hashtable<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::allocator<std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*> >, std::__detail::_Select1st,
std::equal_to<capnp::Type>, capnp::(anonymous namespace)::TypeHash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true,
false, true> >::find(capnp::Type const&) const ()
#4 0x0000000001e483a3 in std::unordered_map<capnp::Type,
capnp::JsonCodec::HandlerBase*, capnp::(anonymous
namespace)::TypeHash, std::equal_to<capnp::Type>,
std::allocator<std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*> > >::find(capnp::Type const&) const ()
#5 0x0000000001e443d5 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#6 0x0000000001e44a6f in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#7 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#8 0x0000000001e45377 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#9 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#10 0x0000000001e45377 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#11 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#12 0x0000000001e45377 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#13 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#14 0x0000000001e45618 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#15 0x0000000001e45a36 in capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#16 0x0000000001e45618 in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type, capnp::JsonValue::Builder) const ()
#17 0x0000000001e43fcc in capnp::JsonCodec::encode(capnp::DynamicValue::Reader,
capnp::Type) const ()
#18 0x00000000009e02c7 in capnp::JsonCodec::encode<Message::Builder
const&> (this=0x7ffff22e1210, value=...) at /home/marc/.../capnp/compat/js
on.h:216
```
That's about all that I can get, even though capnproto is compiled
with DEBUG.
The message trying to be encoded (sorry, I am not sure I can share
the entire set of schemas), is a series of simple objects, which in the
115 s.initIfaceType(1);
The funny part; not initializing it, doesn't make JsonCodec crash.
But initializing it, or initializing it + setting a value (valid one),
produces the crash always.
Valgrind etc... doesn't complain until that point.
I am trying to isolate the problem, to make it reproducible, but I
am not able yet.
Any ideas on this?
Thanks

--
You received this message because you are subscribed to the Google
Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at https://groups.google.com/group/capnproto.

--

You received this message because you are subscribed to the Google
Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/capnproto.

--

You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/capnproto.

[Marc Sune]

Kenton,

Thanks for the help. I've confirmed the issue was the stack size, as I was
suspecting.

Apparently, one external library was creating threads with
PTHREAD_STACK_MIN x2, which is around 32Kbytes. At our application
callback, where we received the event and before calling encode(), ~26K
bytes of the stack were already already consumed. It still sounds a bit too
much ~8K of stack during the encoding. But raising the stack of that thread
to 2Mbytes, solved the problem.

Not sure if this is going to be of any help. Sorry for the fuzz.

Regards
marc

Post by 'Kenton Varda' via Cap'n Proto
Hi Marc,
I would guess each encode() stack frame is on the order of 100 bytes.
Unless your stacks are *really* small, that doesn't seem like it could be
the problem.
It looks like the real problem has something to do with unexpectedly null
pointers appearing in the schema objects. These objects are declared as
const ::capnp::_::RawSchema s_e682ab4cf923a417 = {
0xe682ab4cf923a417, b_e682ab4cf923a417.words, 225, d_e682ab4cf923a417,
m_e682ab4cf923a417,
8, 14, i_e682ab4cf923a417, nullptr, nullptr, { &s_e682ab4cf923a417,
nullptr, nullptr, 0, 0, nullptr }
}
Your latest stack trace shows a case where, somehow, the second field of
this turned out null, which seems impossible.
Is it at all possible that your generated code was created with a
different version of Cap'n Proto compiler vs. the runtime library and/or
headers you are compiling against?
-Kenton

Post by Marc Sune
Kenton,
Is it remotely possible some of the encode() methods are consuming large
amounts of stack? I would need to to get deep into the code, but glancing
over the code I see some parameters that _seem_ to be passed by value, and
few autos which I am _not sure_ on whether they might perform a copy or not.
Marc

Post by Marc Sune
Thanks Kenton,
An update on this. I am leaning to think it is an stack overflow,
although I couldn't confirm 100%. The truth is that, calling the same code
from the main(), with the same exact objects and encoding routine, doesn't
crash.
However, when the encode() method is called from a thread that is
created by an external library, I consistently get a SIGSEGV. I've tried
the typical ulimit / pthread_attr_setstacksize() without much of a success
(not sure why though).
Re-arranging the code so that the encode() is called with less stack (I
cut 2 or 3 frames), I get past the point of libstdc++, but I consistenly
(gdb) bt
#0 0x00000000009d4e46 in capnp::_::DirectWireValue<unsigned int>::get
*(this=0x0)* at /home/marc/target/rootfs/include/capnp/endian.h:80
#1 0x0000000001e64d37 in
capnp::_::WirePointer::target(capnp::_::SegmentReader*) const ()
#2 0x0000000001e6a83c in
capnp::_::WireHelpers::readStructPointer(capnp::_::SegmentReader*,
capnp::_::CapTableReader*, capnp::_::WirePointer const*, capnp::word const*
, int) ()
#3 0x0000000001e5e7e7 in capnp::_::PointerReader::getStruct(capnp::word
const*) const ()
#4 0x0000000001e7fe61 in capnp::_::PointerHelpers<capnp::schema::Node,
(capnp::Kind)3>::get(capnp::_::PointerReader, capnp::word const*) ()
#5 0x0000000001e7f6ec in capnp::ReaderFor_<capnp::schema::Node,
(kind<capnp::schema::Node>)()>::Type
capnp::AnyPointer::Reader::getAs<capnp::schema::Node>() const ()
#6 0x0000000001e7de31 in capnp::schema::Node::Reader
capnp::readMessageUnchecked<capnp::schema::Node>(capnp::word const*) ()
#7 0x0000000001e780db in capnp::Schema::getProto() const ()
#8 0x0000000001e7995f in capnp::EnumSchema::getEnumerants() const ()
#9 0x0000000001e8068f in capnp::DynamicEnum::getEnumerant() const ()
#10 0x0000000001e45c74 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
Really odd... So I think it is not related to capnproto code
Thanks
marc

Post by 'Kenton Varda' via Cap'n Proto
Sorry, I don't really have any ideas here. The stack trace is deep in
STL code for the type handler map, inside find(). If you've registered no
type handlers, that map should be empty. It's hard to imagine how find() on
an empty std::unordered_map could ever segfault...
-Kenton

Post by Marc Sune
Kenton,

Post by 'Kenton Varda' via Cap'n Proto
Hi Marc,
Do you have any custom type handlers registered via addTypeHandler()?
Is it possible that the handler class has gone out-of-scope (been
destroyed) by the time the encoder is executed?

No, I am not using custom handlers. I am just dumping the Builder like
template<typename T>
void dump_msg(T& msg){
try{
capnp::JsonCodec enc;
enc.setPrettyPrint(PPRINT);
auto t = enc.encode(msg);
fprintf(stderr, "MSG: %s\n", t.cStr());
}catch(...){}
}
147 capnp::MallocMessageBuilder builder;
148 auto msg = builder.initRoot<Message>();
//Fill it
254 dump_msg(msg);
So the builder and encoder al valid, I believe in the entire dumping.
Moreover, valgrind would complain before the SEGFAULT if something would be
out of the stack, and the only thing I get is the direct SEGFAULT.
Any thoughts? I will keep trying to isolate the problem
marc
-Kenton

Post by 'Kenton Varda' via Cap'n Proto

Post by Marc Sune
That's about all that I can get, even though capnproto is compiled

with DEBUG.
Or shall I say, it should :/

Post by Marc Sune
Hi guys,
```
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 27946]
0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash,
true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1, capnp::(anonymous
namespace)::TypeHash, true> const&) ()
(gdb) bt
#0 0x0000000001e4b81a in std::__detail::_Hashtable_ebo_helper<1,
capnp::(anonymous namespace)::TypeHash,
true>::_S_cget(std::__detail::_Hashtable_ebo_helper<1, capnp::(anonymous
namespace)::TypeHash, true> const&) ()
#1 0x0000000001e4b11a in
std::__detail::_Hash_code_base<capnp::Type, std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*>, std::__detail::_Select1st,
capnp::(anonymous namespace)::TypeHash, std::__detail::_Mod_range_hashing,
std::__detail::_Default_ranged_hash, true>::_M_h1() const ()
#2 0x0000000001e4a976 in
std::__detail::_Hash_code_base<capnp::Type, std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*>, std::__detail::_Select1st,
capnp::(anonymous namespace)::TypeHash, std::__detail::_Mod_range_hashing,
std::__detail::_Default_ranged_hash, true>::_M_hash_code(capnp::Type
const&) const ()
#3 0x0000000001e4a401 in std::_Hashtable<capnp::Type,
std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>,
std::allocator<std::pair<capnp::Type const, capnp::JsonCodec::HandlerBase*>

, std::__detail::_Select1st, std::equal_to<capnp::Type>, capnp::(anonymous

namespace)::TypeHash, std::__detail::_Mod_range_hashing,
std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy,
std::__detail::_Hashtable_traits<true, false, true> >::find(capnp::Type
const&) const ()
#4 0x0000000001e483a3 in std::unordered_map<capnp::Type,
capnp::JsonCodec::HandlerBase*, capnp::(anonymous namespace)::TypeHash,
std::equal_to<capnp::Type>, std::allocator<std::pair<capnp::Type const,
capnp::JsonCodec::HandlerBase*> > >::find(capnp::Type const&) const ()
#5 0x0000000001e443d5 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#6 0x0000000001e44a6f in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#7 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#8 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#9 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#10 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#11 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#12 0x0000000001e45377 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#13 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#14 0x0000000001e45618 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#15 0x0000000001e45a36 in
capnp::JsonCodec::encodeField(capnp::StructSchema::Field,
capnp::DynamicValue::Reader, capnp::JsonValue::Builder) const ()
#16 0x0000000001e45618 in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type,
capnp::JsonValue::Builder) const ()
#17 0x0000000001e43fcc in
capnp::JsonCodec::encode(capnp::DynamicValue::Reader, capnp::Type) const ()
#18 0x00000000009e02c7 in capnp::JsonCodec::encode<Message::Builder
const&> (this=0x7ffff22e1210, value=...) at
/home/marc/.../capnp/compat/json.h:216
```
That's about all that I can get, even though capnproto is compiled
with DEBUG.
The message trying to be encoded (sorry, I am not sure I can share
the entire set of schemas), is a series of simple objects, which in the
115 s.initIfaceType(1);
The funny part; not initializing it, doesn't make JsonCodec crash.
But initializing it, or initializing it + setting a value (valid one),
produces the crash always.
Valgrind etc... doesn't complain until that point.
I am trying to isolate the problem, to make it reproducible, but I
am not able yet.
Any ideas on this?
Thanks

--
You received this message because you are subscribed to the Google
Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at https://groups.google.com/group/capnproto.

--

You received this message because you are subscribed to the Google
Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/capnproto.

--

You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/capnproto.