Missing "google/protobuf/descriptor.proto".

[BVIVal]

Hi!
I've faced with the problem of compiling 'annotations.proto'-file. It requires an import of "google/protobuf/descriptor.proto" that was not in the repository.

On official repository of 'Protobuf', I've found missing import-file, by the link:
https://github.com/protocolbuffers/protobuf/releases/download/v3.11.1/protobuf-csharp-3.11.1.tar.gz
And, after unzipping, go to 'protobuf-3.11.1\src\google\protobuf\descriptor.proto' and faced another problem - it uses syntax of proto2.

Here is the question:
How did you solve this issue? Please, tell me, I've stuck with my project for 2 days...
By my little experience, gRPC in C# only works with proto3:(

[MAnyKey]

Hi

Could you please share the command that you are trying to run? What version of protoc are you using?
I'm not very well versed in C# protobuf compilation. I've tried this command and it worked just fine:

protoc --proto_path=../cloudapi/ --proto_path=../cloudapi/third_party/googleapis/ --csharp_out=. --grpc_out=. ../cloudapi/yandex/cloud/compute/v1/*.proto --plugin=protoc-gen-grpc=$(which grpc_csharp_plugin)

Note that I'm using protobuf 3.7 and grpc 1.25.0 from Homebrew on Mac OS.

Moreover, @mmarinchenko shared working configuration for C# projects: #1 (comment)
See if it helps you.

[BVIVal]

Hi

Could you please share the command that you are trying to run? What version of protoc are you using?
I'm not very well versed in C# protobuf compilation. I've tried this command and it worked just fine:

protoc --proto_path=../cloudapi/ --proto_path=../cloudapi/third_party/googleapis/ --csharp_out=. --grpc_out=. ../cloudapi/yandex/cloud/compute/v1/*.proto --plugin=protoc-gen-grpc=$(which grpc_csharp_plugin)

Note that I'm using protobuf 3.7 and grpc 1.25.0 from Homebrew on Mac OS.

Moreover, @mmarinchenko shared working configuration for C# projects: #1 (comment)
See if it helps you.

I'm using nuget packages: 'Google.Protobuf 3.11.1', 'Grpc 2.25.0', 'Grpc.Net.Client' and 'Grpc.Tools', like in this video: https://www.youtube.com/watch?v=QyxCX2GYHxk

For every proto-file, I'm setting 'Protobuf compiler' in build Actions.
So, I'm not using any command, because I don't know how to use it and in tutorials, they're not using it either.

[mmarinchenko]

Hi @BVIVal!

You do not need to download or copy descriptor.proto file into your project because it is a part of Grpc.Tools package. You may check this by go to <USER_HOME>\.nuget\packages\grpc.tools\<VERSION>\build\native\include\google\protobuf directory. It's ok that it use proto2 syntax.

I'll try to help you if you attach your csproj file which I may use to reproduce your problem (I have no will to watch 1 hour video trying to understand what exactly you are doing :) ).

[mmarinchenko]

@MAnyKey

BTW, you have 2 unused includes in YC API:

1>yandex/cloud/mdb/mysql/v1alpha/user.proto(5,1): warning : warning: Import google/protobuf/wrappers.proto but not used.
1>yandex/cloud/mdb/mysql/v1/user.proto(5,1): warning : warning: Import google/protobuf/wrappers.proto but not used.

[mmarinchenko]

@MAnyKey

Do you plan to add tags representing api version to repository?

[BVIVal]

Thank you @mmarinchenko !

The code of *.csproj-file:
GrpcClient.txt

[mmarinchenko]

@BVIVal

I downloaded your file and make the following changes.

1. I have only VS 2017, so I changed target framework from netcoreapp3.1 to netcoreapp2.1 and commented out Grpc.Net.Client package reference.

2. You have 2 protos in the project file which are not from this repository, so I commented them out too:

• Protos\third_party\googleapis\google\http.proto
• Protos\third_party\googleapis\google\protobuf\descriptor.proto

---

After that I tried to build the project and it failed with the following errors:

• 1>google/api/http.proto : error : File not found.
• 1>Protos/third_party/googleapis/google/api/annotations.proto(19,1): error : Import "google/api/http.proto" was not found or had errors.
• 1>Protos/third_party/googleapis/google/api/annotations.proto(30,3): error : "HttpRule" is not defined.

While compiling annotations.proto the protoc compiler cannot find google/api/http.proto (not the google/protobuf/descriptor.proto). This is because the actual location of google/api/http.proto is Protos/third_party/googleapis/google/api/http.proto.

To solve this particular error you may specify ProtoRoot attribute:

• <Protobuf Include="Protos\third_party\googleapis\google\api\annotations.proto" ProtoRoot="Protos\third_party\googleapis" GrpcServices="Client" />

But if you do this then the yandex protos will not compile:

• 1>yandex/cloud/validation.proto : error : File not found.
• 1>Protos/yandex/cloud/access/access.proto(5,1): error : Import "yandex/cloud/validation.proto" was not found or had errors.

And vice versa. This is exactly the issue #1 which I reported earlier: yandex protos and google protos are located in different directories but Grpc.Tools package allows to specify only 1 common proto root (in addition to standard include directory with descriptor.proto and others).

---

So first of all I recommend you to carefully read the official documentation of Grpc.Tools here. Based on that you have 2 options:

1. Copy google protos from Protos/third_party/googleapis to Protos directory and specify ProtoRoot attribute as Protos. Then you'll end up with something like that:

• <Protobuf Include="Protos\google\api\annotations.proto" ProtoRoot="Protos" GrpcServices="Client" />

ProtoRoot attribute actually required to be specified only once because it has global scope. So you may just update it after all protobuf includes in the same <ItemGroup>:

• <Protobuf Update="Protos\**\*.proto" ProtoRoot="Protos">

2. Call protoc directly. The protoc itself allows to specify many include paths (check the command used by @MAnyKey above). This can be achieved in project configuration using MSBuild target:

<Target Name="MyProtoCompile"  BeforeTargets="BeforeBuild">
    <PropertyGroup>
        <ProtoCCommand>$(Protobuf_ProtocFullPath) --plugin=protoc-gen-grpc=$(gRPC_PluginFullPath)  -I $(Protobuf_StandardImportsPath) --proto_path=Protos --proto_path=Protos/third_party/googleapis --csharp_out=Messages --grpc_out=Services --grpc_opt=client Protos/**/*.proto</ProtoCCommand>
    </PropertyGroup>
    <Message Importance="high" Text="$(ProtoCCommand)" />
    <Exec Command="$(ProtoCCommand)" />
</Target>

I chose first option for myself, so I cannot guarantee that the second one will work.

---

I wrote some MSBuild magic to copy all protos to temporary _cache directory with one common ProtoRoot and compile them from this temporary directory (which added to my .gitignore of course).

Another issue of Grpc.Tools which annoyed me is auto-compile feature. Each time I change something in the protos it triggers recompile. So I disabled this by set <DisableProtobufDesignTimeBuild> to true and wrote more MSBuild magic to compile protos only then I build my project or solution. You can check this in example attached to issue #1 last comment.

Good luck!

[HavenDV]

I found a solution here - https://github.com/mifopen/YandexCloudDotNet
Thank you @mifopen

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
    <Protobuf_NoWarnMissingExpected>true</Protobuf_NoWarnMissingExpected>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Grpc" Version="2.25.0" /> 
    <PackageReference Include="Grpc.Tools" Version="2.25.0" PrivateAssets="all" />
    <PackageReference Include="Google.Protobuf" Version="3.11.1" />
  </ItemGroup>

  <ItemGroup>
    <Protobuf Include="cloudapi/**/*.proto" ProtoRoot="cloudapi/third_party/googleapis;cloudapi" Access="internal" OutputDir="generated/%(RelativeDir)" GrpcServices="none" CompileOutputs="false" />
    <Protobuf Update="cloudapi/**/*_service.proto" GrpcServices="Client" CompileOutputs="false" />
  </ItemGroup>

</Project>

[mmarinchenko]

Thanks, @HavenDV (and @mifopen)!

So ProtoRoot can contain multiple directories separated by semicolon... Is it documented somewhere?

[HavenDV]

I finalized the project using the DisableProtobufDesignTimeBuild property and the code from the posts above. This project does not produce errors during the first build (if there is already code that uses some services). Maybe this will help someone

End Code(Updated)

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
    <Protobuf_NoWarnMissingExpected>true</Protobuf_NoWarnMissingExpected>
    <DisableProtobufDesignTimeBuild>true</DisableProtobufDesignTimeBuild>
    <ProtoOutputDir>generated</ProtoOutputDir>
    <CompileProtoOutputs>false</CompileProtoOutputs>
    <CompileProtoOutputs Condition="!Exists('$(ProtoOutputDir)')">true</CompileProtoOutputs>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Grpc" Version="2.25.0" />
    <PackageReference Include="Grpc.Tools" Version="2.25.0" PrivateAssets="all" />
    <PackageReference Include="Google.Protobuf" Version="3.11.1" />
  </ItemGroup>

  <ItemGroup>
    <Protobuf Include="cloudapi/**/*.proto" ProtoRoot="cloudapi/third_party/googleapis;cloudapi" Access="internal" OutputDir="$(ProtoOutputDir)/%(RelativeDir)" GrpcServices="none" CompileOutputs="$(CompileProtoOutputs)" />
    <Protobuf Update="cloudapi/**/*_service.proto" GrpcServices="Client" CompileOutputs="$(CompileProtoOutputs)" />
  </ItemGroup>

  <Target Name="Generate" BeforeTargets="BeforeBuild" Condition="!Exists('$(ProtoOutputDir)')">
    <MSBuild Projects="$(MSBuildProjectFile)"
             Properties="DisableProtobufDesignTimeBuild=false"
             Targets="Protobuf_Compile"
             UnloadProjectsOnCompletion="true"
    />
  </Target>
  <Target Name="CleanGenerated" BeforeTargets="BeforeClean" Condition="Exists('$(ProtoOutputDir)')">
    <RemoveDir Directories="$(ProtoOutputDir)" />
  </Target>

</Project>

[HavenDV]

@mmarinchenko

So ProtoRoot can contain multiple directories separated by semicolon... Is it documented somewhere?

https://github.com/grpc/grpc/search?q=ProtoRoot&unscoped_q=ProtoRoot
Judging by the source code, it seems that this behavior is due to the internals of Microsoft.Build.Framework.ToolTask, because the source code does not contain checks that there can be several paths

[mmarinchenko]

Another thanks, @HavenDV!

I'll try your approach in my project next week and share my experience.

[mmarinchenko]

@BVIVal

I've tested ProtoRoot attribute with multiple directories separated by semicolon in my project and may confirm that it's working.

You may find updated version of my sample at issue #1's new comment. Feel free to use it as you wish.

---

@HavenDV

Your project from the comment above has several issues.

• There is no reason to enable Protobuf_NoWarnMissingExpected property when you include all *.proto files without grpc service compilation and then seperately update *_service.proto files to compile grpc services (docs).
• Grpc.Tools package is design-time only. So its assets must be excluded from the runtime dependencies to avoid unnecessary increase in size of product distribution (dotnet publish deploy all package assets by default):

    <PackageReference Include="Grpc.Tools" Version="2.25.0">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
    </PackageReference>

• In your example proto files compilation depends on the existence of ProtoOutputDir. So when proto files are updated you have to always clean the project first or use Rebuild instead of Build.
  Moreover if you add new proto file then you'll need to reload the project before Rebuild. This is because the <Protobuf Include/Update> is static and adds files only on project load. That's why I call MSBuild task with same project file as a parameter on BeforeBuild target without condition in my example:

  <Target Name="CompileProtoSources" BeforeTargets="BeforeBuild">
    <MSBuild Projects="$(MSBuildProjectFile)" Properties="DisableProtobufDesignTimeBuild=false" Targets="Protobuf_Compile" UnloadProjectsOnCompletion="true" />
    <Message Text="Protobuf sources compilation is finished: $(MSBuildProjectDirectory)\$(GrpcOutputDirectory)" />
  </Target>

[HavenDV]

@mmarinchenko,
Thank you for the comments.

1. I applied Protobuf_NoWarnMissingExpected to avoid the following messages after the first compilation
   

2. If I understood everything correctly, it’s simple enough to do so:

<PackageReference Include="Grpc.Tools" Version="2.25.0" PrivateAssets="all" />

so the dependency is available only to this project

3. Yes, this is done on the assumption that .proto files will rarely be updated and there is no problem making Rebuild. As for calling the MSBuild task - I do it too, so there should be no problems with updated files

[mmarinchenko]

@HavenDV, thanks for the info.

1. Hm, interesting. I failed to reproduce these warnings with GrpcServices="None" in <Protobuf Include>. In my environment this only happens if GrpcServices is set to something else (I tested with Client).

2. Yes, you're right. It seems I messed up your finalized project sample with your previous sample because of the spoiler. Or just missed the attribute :) Sorry for that.

3. Based on my experiences in large projects some devs forget about such manual things. But of course this may be documented in something like "Update Y.Cloud API checklist".