[capnproto] First (alpha) release of a Haskell implementation.

[Ian Denhardt]

Hey All,

I've been working on a Haskell implementation of Cap'N Proto, and the
other day finally tagged a first release and published the package:

https://hackage.haskell.org/package/capnp

The source repo is here:

https://github.com/zenhack/haskell-capnp

It's alpha quality, but serialization works (no rpc yet). The API will
probably change, to accommodate more features and performance
improvements, as part of cleanup & simplification that I know needs to
happen, and in response to user feedback.

I look forward to feedback. I don't know how many Haskellers are
subscribed to this list, but hopefully more soon.

-Ian

['Kenton Varda' via Cap'n Proto]

Nice! Let me know when you want this added to the "other languages" page.

How does zero-copy work out in Haskell? It seems like you can't really do
zero-copy writes in a purely-functional way, right?

-Kenton

Post by Ian Denhardt
Hey All,
I've been working on a Haskell implementation of Cap'N Proto, and the
https://hackage.haskell.org/package/capnp
https://github.com/zenhack/haskell-capnp
It's alpha quality, but serialization works (no rpc yet). The API will
probably change, to accommodate more features and performance
improvements, as part of cleanup & simplification that I know needs to
happen, and in response to user feedback.
I look forward to feedback. I don't know how many Haskellers are
subscribed to this list, but hopefully more soon.
-Ian
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/capnproto.

[Ian Denhardt]

Quoting Kenton Varda (2018-08-27 20:41:57)

Post by 'Kenton Varda' via Cap'n Proto
Nice! Let me know when you want this added to the "other languages" page.

Go right ahead.

Post by 'Kenton Varda' via Cap'n Proto
How does zero-copy work out in Haskell? It seems like you can't really
do zero-copy writes in a purely-functional way, right?

The low-level mechanisms in the library aren't ultimately that different
than in most implementations. It's possible to get locally-mutable state
in Haskell without breaking the global invariants via a trick with the
type system that basically emulates Rust's lifetimes -- so the mutable
variables can't be used outside of a certain scope. Naively this
involves a single copy on the way out for the data you want to keep, but
e.g. the vector library provides a create[1] function, which lets you
write code like (to paraphrase & reformat the example on that page):

immutableResult = create $ do
vec <- new 2
write vec 0 'a'
write vec 1 'b'
return vec

So you return a mutable value, which becomes unavailable when you leave
the scope, and you get an immutable one back.

Internally, create calls unsafeFreeze to cast from mutable -> immutable, but
provides an API that can't directly be used in ways that violate
referential transparency.

---

The harder questions were all around how to package stuff up in a way
that was actually comfortable to use; there are worse things than writing
Haskell like it's C, but I wanted to come up with something a little
higher level for common cases.

Right now there's a split high-level/low-level set of interfaces where the
high level ones basically ignore all of the novel properties of the wire
format, parse the thing up front, and expose an idiomatic Haskell data
type. This is not unlike the Go implementation's "pogs" package, except
that there's support in the code generator.

I have some ideas on other spots in the design space that get you a
larger share of the benefits of the wire format without much compromise
of programmer convenience, but they're not implemented yet. My main
interest is really in RPC, which is a much less weird fit for the
language anyway.

Ironically, *read* support was the harder direction to design for,
because of the need to track the traversal limit and deal with errors
in the midst of traversing the data. The current design again just
has a split, where you can either parse the whole thing up front
and forget about it, or write very imperative-looking code with
explicit error checks everywhere.

I have some ideas for how to provide most of the convenience of
the high-level API while still being able to do random access/only
access part of the message, and the API surface for them is pretty
tiny, so they should slot in nicely, but it hasn't happened yet.

[1]: https://hackage.haskell.org/package/vector-0.12.0.1/docs/Data-Vector.html#v:create

['Kenton Varda' via Cap'n Proto]

Post by Ian Denhardt
Quoting Kenton Varda (2018-08-27 20:41:57)

Post by 'Kenton Varda' via Cap'n Proto
Nice! Let me know when you want this added to the "other languages" page.

Go right ahead.

Done (with the 0.7 release).

Post by Ian Denhardt

Post by 'Kenton Varda' via Cap'n Proto
How does zero-copy work out in Haskell? It seems like you can't really
do zero-copy writes in a purely-functional way, right?

The low-level mechanisms in the library aren't ultimately that different
than in most implementations. It's possible to get locally-mutable state
in Haskell without breaking the global invariants via a trick with the
type system that basically emulates Rust's lifetimes -- so the mutable
variables can't be used outside of a certain scope. Naively this
involves a single copy on the way out for the data you want to keep, but
e.g. the vector library provides a create[1] function, which lets you
immutableResult = create $ do
vec <- new 2
write vec 0 'a'
write vec 1 'b'
return vec
So you return a mutable value, which becomes unavailable when you leave
the scope, and you get an immutable one back.
Internally, create calls unsafeFreeze to cast from mutable -> immutable, but
provides an API that can't directly be used in ways that violate
referential transparency.

Oh neat, I didn't know this existed.

Post by Ian Denhardt
---
The harder questions were all around how to package stuff up in a way
that was actually comfortable to use; there are worse things than writing
Haskell like it's C, but I wanted to come up with something a little
higher level for common cases.
Right now there's a split high-level/low-level set of interfaces where the
high level ones basically ignore all of the novel properties of the wire
format, parse the thing up front, and expose an idiomatic Haskell data
type. This is not unlike the Go implementation's "pogs" package, except
that there's support in the code generator.

This makes a lot of sense. I still plan to add "POCS" support to the C++
implementation at some point.

Post by Ian Denhardt
I have some ideas on other spots in the design space that get you a
larger share of the benefits of the wire format without much compromise
of programmer convenience, but they're not implemented yet. My main
interest is really in RPC, which is a much less weird fit for the
language anyway.
Ironically, *read* support was the harder direction to design for,
because of the need to track the traversal limit and deal with errors
in the midst of traversing the data. The current design again just
has a split, where you can either parse the whole thing up front
and forget about it, or write very imperative-looking code with
explicit error checks everywhere.

Interesting. I would think that for validation errors, at least, you could
return `error` and let that bubble up as needed. But the traversal limit is
indeed rather inherently imperative. I wonder if there's some other
strategy for detecting aliasing that would be more friendly to a
purely-functional approach.

Post by Ian Denhardt
I have some ideas for how to provide most of the convenience of
the high-level API while still being able to do random access/only
access part of the message, and the API surface for them is pretty
tiny, so they should slot in nicely, but it hasn't happened yet.
[1]: https://hackage.haskell.org/package/vector-0.12.0.1/docs/
Data-Vector.html#v:create

[Ian Denhardt]

Quoting Kenton Varda (2018-08-28 21:34:13)

Post by Ian Denhardt
has a split, where you can either parse the whole thing up front
and forget about it, or write very imperative-looking code with
explicit error checks everywhere.
Interesting. I would think that for validation errors, at least, you
could return `error` and let that bubble up as needed.

Looking back I think I may have brain-farted when communicating here.
There actually isn't much explicit error checking, but nonetheless the
need to handle errors kindof infects the whole API in a way that you
can't really ignore even if you want to just let the caller handle it.
You *could* as you say just throw an error and have it bubble up, but
the trouble is it's really hard to reason about *when* stuff gets
evaluated in Haskell (because of laziness), and so throwing unchecked
exceptions from pure code tends to result in really brittle applications.

Right now we end up tracking the possibility of errors in the type system,
and while you can abstract the actual handling of them away to a certain
extent, the resulting code is awkward in the same way promises are
awkward compared to straight-line code.

I do have some ideas on how to improve the situation that I want to
experiment with down the line; I think it is possible to tame the
exception approach, but finding the right API will take some thought &
experimentation.

Post by Ian Denhardt
But the traversal limit is indeed rather inherently imperative.
I wonder if there's some other strategy for detecting aliasing that
would be more friendly to a purely-functional approach.

At some point I plan to experiment an API that lets you just set a timeout
and/or an allocation limit; the nice thing about pure code is there's no
worrying about what happens to state when you kill it.

-Ian