[capnproto] Waiting for a promise from inside a server callback?

[The Cheaterman]

Hi everyone, I hope you're doing goodie!

First, thanks a lot Kenton for all the magic! I want to use it! However,
I'd like to use it from Python if possible hehe, but as you may know
pycapnp is somewhat unmaintained (one thing I'd like to see it have is
support for more than one client when wrapping a socket FD, or support for
UDP connection strings somehow, maybe with udp: or something), and that's
why I'm here instead - I'm probably misunderstanding how to use Capnp RPC
more than anything else.

So, let's get to the point: I'm trying to learn how to use Capnp RPC in
Python, so I made what seemed to me like one of the simplest things to do -
a chat program. I managed to make it work with extremely poor network
architecture where the client polls the server (several times per second)
for new messages to be received. The code is here:
https://github.com/Cheaterman/capnchat/ - the master branch still uses the
"polling" protocol, while the "push_messages" branch attempts to push the
messages directly to the client.

The issue happens here:
https://github.com/Cheaterman/capnchat/blob/push_messages/server.py#L69
(you probably already guessed I was trying something like that from the
title), which attempts to wait on this promise
<https://github.com/Cheaterman/capnchat/blob/push_messages/server.py#L119>,
which corresponds to this part
<https://github.com/Cheaterman/capnchat/blob/push_messages/chatroom.capnp#L23>
of the proto (implemented here in the client
<https://github.com/Cheaterman/capnchat/blob/push_messages/client.py#L70> -
BTW I have no idea if having this ".Server" class on the client is a good
idea).

As you already mentioned in other threads, the reason this doesn't work is
because I can't make the main event loop wait for things (AIUI) because
nothing would be processed in the meantime ; and callbacks are executed in
the event loop. The problem is, if I don't wait() on that promise, it's
never actually executed, so I don't quite know what I should be doing
here... You also mentioned in other threads that I might want to add my
promise to the TaskSet or something? I don't think this is exposed in
Python, but I imagine wrapping it wouldn't be an issue (even with my
limited knowledge of modern C++ and how to use it from Cython).

In any case, thanks in advance for clearing out any misunderstanding I may
have as to how I am supposed to implement that kind of things. If things go
well, I'd like to use Capnp RPC to implement a multiplayer games protocol,
please tell me if that makes sense at all or if I should stop right there
:-P

Thanks for reading!

['Kenton Varda' via Cap'n Proto]

Hi The Cheaterman,

The trick here is that your RPC methods can return a promise rather than a
result. The method call is not considered "done" until the returned promise
completes. If you don't do this, then the method is considered to be done
immediately, and all the promises you created but didn't use are discarded
/ canceled.

In your case you are constructing several promises, one for each client.
So, you also need to join them.

def send(self, message, _context, **kwargs):
message = message.as_builder()
self.chatroom.messages.append(message)
promises = []
for client in self.chatroom.users:
promises.append(client.send(message))
return capnp.join_promises(promises)

Note that if you wanted to do something with the result of the
client.send() calls, you would use .then() to register a callback. You can
see some examples in the calculator server sample here:

https://github.com/capnproto/pycapnp/blob/develop/examples/calculator_server.py
https://github.com/capnproto/pycapnp/blob/develop/examples/calculator.capnp

-Kenton

Post by The Cheaterman
Hi everyone, I hope you're doing goodie!
First, thanks a lot Kenton for all the magic! I want to use it! However,
I'd like to use it from Python if possible hehe, but as you may know
pycapnp is somewhat unmaintained (one thing I'd like to see it have is
support for more than one client when wrapping a socket FD, or support for
UDP connection strings somehow, maybe with udp: or something), and that's
why I'm here instead - I'm probably misunderstanding how to use Capnp RPC
more than anything else.
So, let's get to the point: I'm trying to learn how to use Capnp RPC in
Python, so I made what seemed to me like one of the simplest things to do -
a chat program. I managed to make it work with extremely poor network
architecture where the client polls the server (several times per second)
https://github.com/Cheaterman/capnchat/ - the master branch still uses
the "polling" protocol, while the "push_messages" branch attempts to push
the messages directly to the client.
The issue happens here: https://github.com/Cheaterman/
capnchat/blob/push_messages/server.py#L69 (you probably already guessed I
was trying something like that from the title), which attempts to wait on this
promise
<https://github.com/Cheaterman/capnchat/blob/push_messages/server.py#L119>,
which corresponds to this part
<https://github.com/Cheaterman/capnchat/blob/push_messages/chatroom.capnp#L23>
of the proto (implemented here in the client
<https://github.com/Cheaterman/capnchat/blob/push_messages/client.py#L70>
- BTW I have no idea if having this ".Server" class on the client is a good
idea).
As you already mentioned in other threads, the reason this doesn't work is
because I can't make the main event loop wait for things (AIUI) because
nothing would be processed in the meantime ; and callbacks are executed in
the event loop. The problem is, if I don't wait() on that promise, it's
never actually executed, so I don't quite know what I should be doing
here... You also mentioned in other threads that I might want to add my
promise to the TaskSet or something? I don't think this is exposed in
Python, but I imagine wrapping it wouldn't be an issue (even with my
limited knowledge of modern C++ and how to use it from Cython).
In any case, thanks in advance for clearing out any misunderstanding I may
have as to how I am supposed to implement that kind of things. If things go
well, I'd like to use Capnp RPC to implement a multiplayer games protocol,
please tell me if that makes sense at all or if I should stop right there
:-P
Thanks for reading!
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/capnproto.

[The Cheaterman]

Hi again Kenton,

Thanks a lot for everything! I apologize to everyone as I just noticed most
of our conversation went private after I inadvertently pressed the
"'respond" button instead of "respond to all" in GMail. The main issue was
solved here by Kenton, but on the client side, I was using a simple
time.sleep() call to slow down my event loop, which I had to convert to a
KJ timer, to make sure the KJ EventLoop is actually being run. Fortunately, this
is also exposed in Python
<https://github.com/capnproto/pycapnp/blob/develop/capnp/lib/capnp.pyx#L1692>!
I currently use it with:

capnp.getTimer().after_delay(.01 * 10 ** 9).wait()

instead of time.sleep(.01), and it works wonders :-)

If I implement more complex things however, I'll need to consider doing
everything in proper asynchronous fashion: Kenton suggested a
UnixEventPort::FdObserver for managing user input, which might not be
exposed to Python (I didn't check yet) ; he also suggested a workaround
where I could have threads communicating with each other using Capnp RPC,
one would be exclusively for input, and the other would receive messages
from both the input thread and server. I believe this second way would work
in Python, so I might implement it in my small chat project
<https://github.com/Cheaterman/capnchat> in the future.

In the meantime, I'm going to push these updates. Thanks a lot to Kenton
for all the help!

Post by 'Kenton Varda' via Cap'n Proto
Hi The Cheaterman,
The trick here is that your RPC methods can return a promise rather than a
result. The method call is not considered "done" until the returned promise
completes. If you don't do this, then the method is considered to be done
immediately, and all the promises you created but didn't use are discarded
/ canceled.
In your case you are constructing several promises, one for each client.
So, you also need to join them.
message = message.as_builder()
self.chatroom.messages.append(message)
promises = []
promises.append(client.send(message))
return capnp.join_promises(promises)
Note that if you wanted to do something with the result of the
client.send() calls, you would use .then() to register a callback. You can
https://github.com/capnproto/pycapnp/blob/develop/examples/calculator_server.py
https://github.com/capnproto/pycapnp/blob/develop/examples/calculator.capnp
-Kenton

Post by The Cheaterman
Hi everyone, I hope you're doing goodie!
First, thanks a lot Kenton for all the magic! I want to use it! However,
I'd like to use it from Python if possible hehe, but as you may know
pycapnp is somewhat unmaintained (one thing I'd like to see it have is
support for more than one client when wrapping a socket FD, or support for
UDP connection strings somehow, maybe with udp: or something), and that's
why I'm here instead - I'm probably misunderstanding how to use Capnp RPC
more than anything else.
So, let's get to the point: I'm trying to learn how to use Capnp RPC in
Python, so I made what seemed to me like one of the simplest things to do -
a chat program. I managed to make it work with extremely poor network
architecture where the client polls the server (several times per second)
https://github.com/Cheaterman/capnchat/ - the master branch still uses
the "polling" protocol, while the "push_messages" branch attempts to push
the messages directly to the client.
https://github.com/Cheaterman/capnchat/blob/push_messages/server.py#L69
(you probably already guessed I was trying something like that from the
title), which attempts to wait on this promise
<https://github.com/Cheaterman/capnchat/blob/push_messages/server.py#L119>,
which corresponds to this part
<https://github.com/Cheaterman/capnchat/blob/push_messages/chatroom.capnp#L23>
of the proto (implemented here in the client
<https://github.com/Cheaterman/capnchat/blob/push_messages/client.py#L70>
- BTW I have no idea if having this ".Server" class on the client is a good
idea).
As you already mentioned in other threads, the reason this doesn't work
is because I can't make the main event loop wait for things (AIUI) because
nothing would be processed in the meantime ; and callbacks are executed in
the event loop. The problem is, if I don't wait() on that promise, it's
never actually executed, so I don't quite know what I should be doing
here... You also mentioned in other threads that I might want to add my
promise to the TaskSet or something? I don't think this is exposed in
Python, but I imagine wrapping it wouldn't be an issue (even with my
limited knowledge of modern C++ and how to use it from Cython).
In any case, thanks in advance for clearing out any misunderstanding I
may have as to how I am supposed to implement that kind of things. If
things go well, I'd like to use Capnp RPC to implement a multiplayer games
protocol, please tell me if that makes sense at all or if I should stop
right there :-P
Thanks for reading!
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/capnproto.