gRPC name resolution providers

Introduction

We have something special today. It all started with that I updated grpc-java version from 1.43.2 to 1.60.0. Alright, I just updated the dependency and anticipated it working. However, I run into exception on ManagedChannel build at the client side:

ManagedChannel channel = ManagedChannelBuilder.forAddress("localhost", 8081)
	.usePlaintext()
	.build();

Exception I got looks like this:

java.lang.IllegalArgumentException: Address types of NameResolver 'unix' for 'localhost:8081' not supported by transport
	at io.grpc.internal.ManagedChannelImpl.getNameResolver(ManagedChannelImpl.java:750)
	at io.grpc.internal.ManagedChannelImpl.getNameResolver(ManagedChannelImpl.java:771)
	at io.grpc.internal.ManagedChannelImpl.<init>(ManagedChannelImpl.java:635)
	at io.grpc.internal.ManagedChannelImplBuilder.build(ManagedChannelImplBuilder.java:668)
	at io.grpc.ForwardingChannelBuilder2.build(ForwardingChannelBuilder2.java:260)
	at org.example.grpcbase.client.GrpcClient.main(GrpcClient.java:14)

Another modifications:

Exception in thread "main" java.lang.IllegalArgumentException: Could not find a NameResolverProvider for dns:///localhost:8081
	at io.grpc.internal.ManagedChannelImpl.getNameResolver(ManagedChannelImpl.java:741)
	at io.grpc.internal.ManagedChannelImpl.getNameResolver(ManagedChannelImpl.java:771)
	at io.grpc.internal.ManagedChannelImpl.<init>(ManagedChannelImpl.java:635)
	at io.grpc.internal.ManagedChannelImplBuilder.build(ManagedChannelImplBuilder.java:668)
	at io.grpc.ForwardingChannelBuilder2.build(ForwardingChannelBuilder2.java:260)
	at org.example.grpcbase.client.GrpcClient.main(GrpcClient.java:19)

One more:

Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: connect(..) failed: Address family not supported by protocol: /localhost:8081
Caused by: java.net.ConnectException: connect(..) failed: Address family not supported by protocol
        at io.netty.channel.unix.Errors.newConnectException0(Errors.java:155)
        at io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128)
        at io.netty.channel.unix.Socket.connect(Socket.java:312)

And another one:

Exception in thread "main" io.grpc.StatusRuntimeException: UNKNOWN
	at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:271)
	at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:252)
	at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:165)
	at org.example.grpc.base.HelloServiceGrpc$HelloServiceBlockingStub.hello(HelloServiceGrpc.java:157)
	at org.example.grpcbase.client.GrpcClient.main(GrpcClient.java:19)
Caused by: java.nio.channels.UnsupportedAddressTypeException

TL;DR No-Brainer Solution

If you’re looking for the simplest & quickest no-brainer workaround (TL;DR) – here you go:

NameResolverRegistry.getDefaultRegistry().register(new DnsNameResolverProvider());

Add this line before building your ManagedChannel / NettyChannel. If you want elaboration on that or you want to know more correct solution – keep reading further.

Theory

Let’s start with some theory knowledge. Two key concepts here are:

• NameResolvers
• ChannelProviders

NameResolver is a gRPC component responsible for resolving the target address of a service. It translates a logical server address (such as a hostname) into a physical address (such as IP address) that can be used to establish a connection. gRPC has at least two flavors of it:

• DNS NameResolver. This NameResolver uses the Domain Name System to resolve service names to IP addresses. It’s when you create gRPC channel and specify network address like localhost:8080. Whenever you use addresses like that, io.grpc.internal.DnsNameResolverProvider is your choice.

• UDS NameResolver. Unix Domain Sockets (UDS) provide a way for processes on the same machine to communicate with each other using socket addresses that are based on file paths. This can be useful in scenarios where communication needs to occur locally without going through the network stack. it’s not suitable for scenarios where clients and servers reside on different machines. This provider is intended to resolve UDS addresses like this: /tmp/socketname.socket. Whenever you want to you UDS, io.grpc.netty.UdsNameResolverProvider / io.grpc.netty.shaded.<…> are your choices.

There is a concept of NameResolverRegistry. It’s the place when gRPC maintains references to all NameResolvers you have in your classpath. It means that:

• If you specify an address like dns:///localhost:8080, gRPC will search for a NameResolver in NameResolverRegistry, that returns value of dns if calling getDefaultScheme() method.
• If you specify an address like unix:///tmp/socketname.socket, then gRPC will search for a NameResolver in NameResolverRegistry which returns unix as a default scheme.

Here is a getDefaultScheme() for DnsNameResolverProvider:

private static final String SCHEME = "dns";

@Override
public String getDefaultScheme() {
    return SCHEME;
}

Keep in mind, that it might be the case when you don’t specify your scheme explicitly. Consider localhost:8080 or /tmp/socketname.socket. When you don’t specify the scheme, NameResolverRegistry.class uses getDefaultScheme() method in itself:

public synchronized String getDefaultScheme() {
    return defaultScheme;
}

And defaultScheme field is being populated from the NameResolver with the highest priority. Yeah, we have the concept of NameResolver priority. Each NameResolver has its own priority. Let’s look at io.grpc.internal.DnsNameResolverProvider, for instance:

@Override
public int priority() {
    return 5;
}

And UdsNameResolverProvider has a priority of 3. NameResolver with the greatest priority wins. It means that when you get DNS and UDS resolvers registered in your registry, DNS will be the choice for addresses without schema explicitly specified.

If you’re interested in sorting, here you go, you can see this place in io.grpc.NameResolverRegistry:

List<NameResolverProvider> providerList = ServiceProviders.loadAll(
    NameResolverProvider.class,
    getHardCodedClasses(),
    NameResolverProvider.class.getClassLoader(),
    new NameResolverPriorityAccessor()
);

NameResolverPriorityAccessor is a sort of comparator that compares the priority of NameResolvers and chooses one that has greatest priority value.

Returning to our initial exception. You see that we have managed channel creation like this: ManagedChannelBuilder.forAddress("localhost", 8081).

NameResolverRegistry should use DNS NameResolver. However, looking one more time at the exception, you see this: Address types of NameResolver ‘unix’. And you’re right, for some reason (more details on this below), NameResolverRegistry uses UDS NameResolver.

Give or take, now you can see the sense of initial solution: we want to force NameResolverRegistry to use the right NameResolver.

NameResolverRegistry, when first created (under the hood of gRPC library), performs loading of NameResolvers using ServiceLoader JDK feature. If you aren’t familiar with this feature, here you go:

• Imagine plug-in architecture, when you have some common plug-ins interface defined, and its implementations (discrete plug-ins), that you want to connect to your main application in runtime without code modifications.
• Interface, in terms of ServiceLoader feature, is named Service; and its implementations named Service providers.
• Suppose we have Service named Route and two implementations: HomeProvider and GymProvider. We have to have three initial jars: one jar with Route.java interface, one jar with HomeProvider.java class and one jar with GymProvider.java class.
• Finally, you have fourth jar – your application. Connect dependencies from above to it. Now, all you have to do is to call ServiceLoader<Route> routeProviders = ServiceLoader.load(Route.class).
• By making use of this feature, you receive an iterator for all providers you have in your classpath. You can play with it a little – just remove dependency with HomeProvider.java, and you see that ServiceLoader.load() will result just in one provider, and no exception threw.

⚠️ Very important note that you have to create file named as fully qualified class name of your Service under <service-provider-jar-root>/META-INF/services directory, and insert just one line – your Service prover’s fully qualified class name. Examples:

• For GymProvider Service provider
  • Create new file gym-provider/META-INF/services/org.example.service.Route
  • Add this line: org.example.gym.GymProvider
• For HomeProvider Service provider:
  • Create new file home-provider/META-INF/services/org.example.service.Route
  • Add this line: org.example.home.HomeProvider

See llw33-service-loader-example repository in my GitHub if you want to see ServiceLoader in action.

You can see NameResolverRegistry works same way:

List<NameResolverProvider> providerList = ServiceProviders.loadAll(/*...*/);

Going further in io.grpc.ServiceProviders.loadAll() -> getCandidatesViaServiceLoader():

Iterable<T> i = ServiceLoader.load(klass, cl);

Locations for NameResolvers are the following:

• io.grpc.internal.DnsNameResolverProvider is being located under grpc-core dependency
• io.grpc.netty.UdsNameResolverProvider is being located under grpc-natty dependency
• io.grpc.netty.shaded.<...> is being located under grpc-natty-shaded.

Every dependency from above has its own service definition in META-INF/service directory I mentioned earlier:

• Consider this for DNS resolver: GitHub link to sources.
• This for UDS resolver: GitHub link to sources.

As we spoke earlier, NameResolverRegistry should use DNS NameResolver. However, it uses UDS NameResolver. The reason beneath this is that most likely you have fat jar, where Maven/Gradle merges these files into one single file! Therefore, you end up with:

<your-fat-jar-root>/META-INF/services/io.grpc.NameResolverProvider file that obviously has single line inside: io.grpc.netty.UdsNameResolverProvider.

All you need to do is to change this line to (or add a new one) io.grpc.internal.DnsNameResolverProvider. You can do this either manually, or by making use of any Maven plugins suitable for that (ant, assembly, shade), or you can define your own service definition file onto your resource folder, that should override any other ones. Same for Gradle. The process can be named as META-INF service definitions merging.

That’s all. If you want to know more about gRPC transport, what are ChannelProviders and the sense of the word “transport” in initial exception, welcome to the next part of our story.

Useful links

• https://github.com/grpc/grpc-java/issues/9367
• https://github.com/grpc/grpc-java/issues/9387