protobuf-net.Grpc
What is it?
Simple gRPC access in .NET Core 3+ and .NET Framework 4.6.1+ - think WCF, but over gRPC
- Google released gRPC, a cross-platform RPC stack over HTTP/2 using protobuf serialization
- included in the Google bits is
Grpc.Core
, Google’s gRPC bindings for .NET; it has kinks:- the “protoc” codegen tool only offers C# (for .NET) and is proto3 only
- contract-first only
- the actual HTTP/2 bits are in an unmanaged binary
- Microsoft are working on
Grpc.Net
- it uses the same core types - so new or existing code based on
Grpc.Core
can work fine - it can use the “Kestrel” HTTP/2 server bindings, and the new
HttpClient
HTTP/2 client bindings - but it still has the other limitations from
Grpc.Core
- it uses the same core types - so new or existing code based on
So what is protobuf-net.Grpc
?
Unrelated to gRPC; for many years now, protobuf-net
has offered idiomatic
protobuf serialization for .NET; protobuf-net.Grpc
takes the best bits from protobuf-net
and Grpc.Net
and smashes
them together to give you:
- support for the managed “Kestrel” (server) and
HttpClient
(client) HTTP/2 bindings on .NET Core 3 - support for the unmanaged
Grpc.Core
chttp2 client and server bindings - code-first or contract-first
- the “protogen” codegen tool is proto2 and proto3, and offers C# and VB
- and if you have code-first support, you can use any .NET language (tested: C#, VB, F#)
Additionally, it even works with the standard (unmanaged) Grpc.Core
implementation if you are limited to .NET Framework (.NET Framework 4.6.1 or .NET Standard 2.0).
I’m interested in code-first; how do I get started?
0: get your build environment
This walkthrough assumes you have the .NET Core 3.1 SDK or above, and an up-to-date compiler, ideally by having an up-to-date IDE. Note that the tools work on more platforms than just .NET Core (although you’d need to use the unmanaged bindings).
Also: make sure that you are actually using the correct .NET Core SDK (usually via “global.json”).
Note: to avoid common usage errors with protobuf-net and gRPC, consider using protobuf-net.BuildTools
, which
reports problems at build-time.
1: define your data contracts and service contracts
Your service and data contracts can be placed directly in the client/server (see later), or can be in a separate class library. If you use
a separate library, make sure you target net6.0
or above.
As for what they look like: think “WCF”. Data contracts are classes marked with either [ProtoContract]
or [DataContract]
, with individual members
annotated with either [ProtoMember]
or [DataMember]
. The [Proto*]
options are protobuf-net specific and offer fine-grained
control - and require a package-reference to protobuf-net
; the [Data*]
options are part
of the BCL and avoid needing an external reference (if you care), but may need
System.ServiceModel.Primitives
. Key point: protobuf
uses integer tokens to identify members (not names), so you need to tell the library how to map them (positive integers, unique per type:
[DataContract]
public class MultiplyRequest
{
[DataMember(Order = 1)]
public int X { get; set; }
[DataMember(Order = 2)]
public int Y { get; set; }
}
[DataContract]
public class MultiplyResult
{
[DataMember(Order = 1)]
public int Result { get; set; }
}
Object models can be arbitrarily deep and complex (including lists, arrays, etc), but should be trees (not graphs).
Service contracts are interfaces marked with [ServiceContract]
(from System.ServiceModel
) or [Service]
(protobuf-net.Grpc inbuilt). You can optionally specify the gRPC service name, otherwise it’ll use
reasonable convention-based defaults. Individual RPC calls are methods, which can optionally be marked with [OperationContract]
(from System.ServiceModel
) or [Operation]
(protobuf-net.Grpc inbuilt) to control
the name. There is also [SubService]
, which is used as below.
Interface inheritance works, with 2 possible scenarios:
- Inherited interfaces as distinct routable services
Consider:
[Service("foo")]
interface IFoo : IBar
{
Task A();
}
[Service("bar")]
interface IBar
{
Task B();
}
Here, the bindings are /foo/A
and /bar/B
. A client can be constructed for IBar
by itself, since IBar
is routable (via /bar/
) - or a client can be constructed for IFoo
, which will route A()
via /foo/A
and B()
via /bar/B
.
If there were additional services that also inherited IBar
, they could not be configured as side-by-side independent implementations of B
, since all uses would want to route via /bar/
.
- Inherited interfaces as composition
Consider:
[Service("foo")]
interface IFoo : IBar
{
Task A();
}
[SubService]
interface IBar
{
Task B();
}
Here, the bindings are /foo/A
, /foo/B
. The methods from IBar
are lifted upwards and form part of the IFoo
service. As a consequence, a client cannot be constructed for IBar
in isolation, as
IBar
is not routable without knowing the top-level service to use. The upside of this, however, is that we can add additional services with the same common API, and route them independently. For example, we can add:
[Service("blap")]
interface IBlap : IBar
{
Task C();
}
This adds the bindings /blap/C
and /blap/B
, so now we have two completely independent routable implementations of B()
. This is especially useful for generic scenarios, common service-level infrastructure APIs, repository APIs, etc.
Call types
In gRPC, there are 4 types of call available:
- unary: the client sends one input; the server sends one output (classic request/response)
- client-streaming: the client can send a sequence of inputs; the server sends one output
- server-streaming: the client sends one input; the server sends a sequence of outputs
- duplex: the client and server can arbitrarily send disconnected sequences at each-other
Let’s start with unary; a simple example there might be:
[ServiceContract(Name = "Hyper.Calculator")] // or [Service("Hyper.Calculator")]
public interface ICalculator
{
ValueTask<MultiplyResult> MultiplyAsync(MultiplyRequest request);
}
We’re using ValueTask<T>
here, but the library also supports Task<T>
and simple synchronous T
(but: please prefer asynchronous when possible). Sometimes,
you don’t actually have data to send in one or both directions; in regular gRPC you’d typically use .google.protobuf.Empty
as a nil token here, but
we don’t need to do that here - we can just have a parameterless method (MultiplyAsync()
) and/or return void
, ValueTask
or Task
. The library understands
what you intend. Additionally, since you may want to specify or query deadlines, credentials, cancellation, or send/receive additional headers, trailers, etc, there is a CallContext
type that expresses this intent, which can be included as an additional parameter after the data parameter. Since you don’t always need this, it is useful
to make this an optional parameter, i.e.
[ServiceContract(Name = "Hyper.Calculator")]
public interface ICalculator
{
ValueTask<MultiplyResult> MultiplyAsync(MultiplyRequest request, CallContext context = default);
}
If you wish to allow optional cancellation in a general purpose way, and do not require access to headers/trailers/etc, then you can use CancellationToken
in
place of CallContext
:
[ServiceContract(Name = "Hyper.Calculator")]
public interface ICalculator
{
ValueTask<MultiplyResult> MultiplyAsync(MultiplyRequest request, CancellationToken cancellationToken = default);
}
If you want to use client/server-streaming or duplex communication, then instead of using T
, Task<T>
etc, you can use IAsyncEnumerable<T>
for
either the data parameter or the return type. For example, we could subscribe to a speaking clock via:
[ServiceContract]
public interface ITimeService
{
IAsyncEnumerable<TimeResult> SubscribeAsync(CallContext context = default);
}
[ProtoContract]
public class TimeResult
{
[ProtoMember(1, DataFormat = DataFormat.WellKnown)]
public DateTime Time { get; set; }
}
The IAsyncEnumerable<TimeResult>
is a server-streaming sequence of values; the DataFormat.WellKnown
here tells protobuf-net
to use the .google.protobuf.Timestamp
representation
of time, which is recommended if you may need to work cross-platform (for legacy reasons, protobuf-net defaults to a different library-specific layout that pre-dates the
introduction of .google.protobuf.Timestamp
). It is recommended to use DataFormat.WellKnown
on DateTime
and TimeSpan
values whenever possible.
2: implement the server
- Create an ASP.NET Core Web Application targeting
net8.0
, and add a package references toprotobuf-net.Grpc.AspNetCore
(and a project/package reference to your data/service contracts if necessary). Note that the gRPC tooling can run alongside other services/sites that your ASP.NET application is providing. - in
CreateHostBuilder
, make sure you are usingWebHost
, and enable listening onHttpProtocols.Http2
; seeProgram.cs
- in
ConfigureServices
, callservices.AddCodeFirstGrpc()
; seeStartup.cs
- define a class that implements your service contract, i.e.
class MyTimeService : ITimeService
- in
Configure
, callendpoints.MapGrpcService<TService>()
for each of your services; seeStartup.cs
So what might our services look like? Let’s start with our simple calculator; that might be synchronous at the server, which is why ValueTask<T>
can be useful; consider:
public class MyCalculator : ICalculator
{
ValueTask<MultiplyResult> ICalculator.MultiplyAsync(MultiplyRequest request)
=> new ValueTask<MultiplyResult>(new MultiplyResult { Result = request.X * request.Y });
}
Very simple. Note that a service type can implement any number of interfaces - any that are marked suitable as [ServiceContract]
will be considered. Note also that the usual
ASP.NET dependency injection works, so if you need additional services you can request them via the constructor of the service type.
How about something more advanced, like our time service? This is a bit more subtle:
public class MyTimeService : ITimeService
{
public IAsyncEnumerable<TimeResult> SubscribeAsync(CallContext context = default)
=> SubscribeAsync(context.CancellationToken);
private async IAsyncEnumerable<TimeResult> SubscribeAsync([EnumeratorCancellation] CancellationToken cancel)
{
while (!cancel.IsCancellationRequested)
{
await Task.Delay(TimeSpan.FromSeconds(10), cancel);
yield return new TimeResult { Time = DateTime.UtcNow };
}
}
}
Here we’re using an asynchronous iterator block (yield return
) to generate a new result every ten seconds until cancellation. Note
that we’re also using the [EnumeratorCancellation]
syntax which helps the C# compiler create an iterator block that will behave
correctly. Cancellation is supported by the core gRPC framework, and is surfaced via CancellationToken
in a familiar way.
Now all we need to do is use dotnet run
, and we should be in business with our gRPC server:
info: ProtoBuf.Grpc.Server.ServicesExtensions.CodeFirstServiceMethodProvider[0]
Server_CS.MyCalculator implementing service Hyper.Calculator (via 'ICalculator') with 1 operation(s)
dbug: Grpc.AspNetCore.Server.Model.Internal.ServiceRouteBuilder[1]
Added gRPC method 'MultiplyAsync' to service 'Hyper.Calculator'. Method type: 'Unary', route pattern: 'Hyper.Calculator/Multiply'.
info: ProtoBuf.Grpc.Server.ServicesExtensions.CodeFirstServiceMethodProvider[0]
Server_CS.MyTimeService implementing service MegaCorpTimeService (via 'ITimeService') with 1 operation(s)
dbug: Grpc.AspNetCore.Server.Model.Internal.ServiceRouteBuilder[1]
Added gRPC method 'SubscribeAsync' to service 'MegaCorpTimeService'. Method type: 'ServerStreaming', route pattern: 'MegaCorpTimeService/Subscribe'.
...
Now listening on: http://localhost:10042
(note I’m using http, not https for this example; gRPC supports TLS and authentication - it just isn’t something I’m covering)
2: implement the client
OK, we have a working server; now let’s write a client. This is much easier, in fact. Let’s create a .NET console application targeting net8.0
,
and add a package reference to protobuf-net.Grpc
. Note that by default, HttpClient
only wants to talk HTTP/2 over TLS, so we first
need to twist it’s arm a little; then we can very easily create a client to our services at our base address; let’s start by doing some maths:
static async Task Main()
{
GrpcClientFactory.AllowUnencryptedHttp2 = true;
using (var channel = GrpcChannel.ForAddress("http://localhost:10042"))
{
var calculator = channel.CreateGrpcService<ICalculator>();
var result = await calculator.MultiplyAsync(new MultiplyRequest { X = 12, Y = 4 });
Console.WriteLine(result.Result);
}
}
If we use dotnet run
, unsurprisingly we see:
48
So let’s do something more exciting and consume our time service for, say, a minute:
var clock = channel.CreateGrpcService<ITimeService>();
var cancel = new CancellationTokenSource(TimeSpan.FromMinutes(1));
var options = new CallOptions(cancellationToken: cancel.Token);
await foreach(var time in clock.SubscribeAsync(new CallContext(options)))
{
Console.WriteLine($"The time is now: {time.Time}");
}
and now we see (with the lines appearing every 10 seconds, not all at once):
48
The time is now: 17/06/2019 18:44:43
The time is now: 17/06/2019 18:44:53
The time is now: 17/06/2019 18:45:03
The time is now: 17/06/2019 18:45:13
The time is now: 17/06/2019 18:45:23
As you would expect, IAsyncEnumerable<T>
works similarly at the server, exposing values sent by the client as they
arrive. In both cases, the thread doesn’t block while waiting for work - the await
here ensures that this is
implemented using a continuation-based model.
Did you mention that it works on .NET Framework too?
Yes; see protobuf-net.Grpc.Native
; to create a client, we start off from via a Channel
(the regular wrapper around the unmanaged gRPC API that Grpc.Core
uses):
var channel = new Channel("localhost", 10042, ChannelCredentials.Insecure);
try
{
var calculator = channel.CreateGrpcService<ICalculator>();
await Test(calculator);
}
finally
{
await channel.ShutdownAsync();
}
You can also implement servers just like we did for ASP.NET - we create a class that implements the service-contract interface, but hosting is provided by Grpc.Core
via the native Server
type:
static async Task Main()
{
const int port = 10042;
Server server = new Server
{
Ports = { new ServerPort("localhost", port, ServerCredentials.Insecure) }
};
server.Services.AddCodeFirst(new MyServer());
server.Start();
Console.WriteLine("server listening on port " + port);
Console.ReadKey();
await server.ShutdownAsync();
}
It is the server’s .Services
collection that acts as the extension point here; by passing in the instance of our server (MyServer
), the library registers the
services just like it would have done within ASP.NET Core, but now using the native gRPC libraries.
Summary
This is just a very brief introduction to what you can do with protobuf-net and gRPC using protobuf-net.Grpc; if you want to play with it, please feel free to do so. Log issues, make suggestions, etc. Have fun.
And if this could save you a ton of time, you’re always welcome to