Protocol Buffers implementation in C

This is an experimental first cut at adding proto3 support and I have not tested it further than porting to proto3 the testcase. Before pushing this further I would like to have some feedback on whether I am on the right track.

As far as I understand protobuf-c already has pretty much everything needed once it is built using a new version of protibuf itself. The only missing thing is that in proto3 all fields are optional and having to manually set has_foo is inconvenient.

This patch special cases the proto3 syntax files so that structs for the bytes, enum and primitive fields do not emit the has_ field.

It also adds PROTOBUF_C_LABEL_NONE to the label enum that is used for proto3 fields. When a fields has this label, the quantifier is not consulted and instead the field is packed/unpacked depending on whether it has a value different from NULL/0.

The patch changes the generated-code test to proto3 as a quick test, clearly this should not be part of the final patch.

The patch also slightly refactors packing/unpacking optional fields so that the internal functions take a boolean istead of a pointer to a boolean: casting in the caller seems cleaner to me... this part can be dropped or split out in its own preliminary patch.

I think I found a bug using the current code with proto3 syntax. I'm using a protobuf with two layers of nested repeated submessages, in other words I have a subsubmessage. I can populate the fields just fine, but packing the buffer turns all the subsubmessage values to 0. The other fields in the submessage are sent fine. Switching back to proto2 fixes the issue.

if I have a message with one of like

message Wrapper { oneof msg { MsgA msg_a = 1; MsgB msg_b = 2; } }

How can I get the offset at which I can write my message A or B?

In other words, if I do

desc = protobuf_c_message_descriptor_get_field_by_name(descriptor, "msg_a"); I can get desc->offset

But this requires knowing that the field is called "msg_a". Is there any way to obtain the same offset only knowing that the oneof is called "msg"?

Basically the use case I have is that I have many wrapper messages that contain "oneof msg", but the actual fields inside the oneof change. I would like to be able to write a generic function that gives me the "unwrapped" message

I have separated the Visual Studio build errors from cmake (https://github.com/protobuf-c/protobuf-c/pull/155).

Here is the patch that allows compilation on Windows. It fixes 3 different (but simple) errors with following reasons:

• use _inline for non-c99 msvc compiler
• use char* instead of void* in pointer shifting
• on non-gnuc family compilers DEPRECATED macros was undefined

(see also https://github.com/protobuf-c/protobuf-c/issues/168)

It'd be nice to have a way to unpack to a struct on the stack, rather than require an allocator. This might not be practical for protos with strings or similar, but should be fine for number-only data.

google::protobuf::message::Reflaction is not exist namespace.

I try to ubuntu build. But namespace is invalid, so can't build.

I changed correct namespace

There were new changes in https://github.com/google/protobuf.git. Because of that protobuf-c install is getting failed. Error faced as below:

checking google/protobuf/compiler/command_line_interface.h usability... no checking google/protobuf/compiler/command_line_interface.h presence... no checking for google/protobuf/compiler/command_line_interface.h... no ERROR: Installing protobuf-c library. Please check env_protobuf.log

The spec talks about "empty string" and other languages like C# return a zero length string and not null. This is useful because when moving from proto2's "required string" to a proto3's plain string we will be guaranteed that we never get a null pointer. The tradeoff is adding a special case to the library but avoiding a lot of null checks in the calling code. The current code is already special casing "" in pack_string.

step1:

./configure CFLAGS=-m64 CXXFLAGS=-m64 LDFLAGS=-m64^C

step 2: make

Get error during make stage: GEN t/test.pb-c.c terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc /bin/sh: line 1: 4337 Aborted (core dumped) ./protoc-c/protoc-c -I. --c_out=. ./t/test.proto

GEN t/test.pb-c.c terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc /bin/sh: line 1: 4337 Aborted (core dumped) ./protoc-c/protoc-c -I. --c_out=. ./t/test.proto

GEN t/test.pb-c.c terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc /bin/sh: line 1: 4337 Aborted (core dumped) ./protoc-c/protoc-c -I. --c_out=. ./t/test.proto

I like to use protobufs as a config file format so the ability to parse text format protobufs would be nice. It's also nice to be able to dump them to text.

I've been using the ProtobufCMessageDescriptor and ProtobufCFieldDescriptor types and they seem like they'll be useful in that and I have the basics of a text format generator now. A parser should be relatively easy.

Would there be any interest in this code for libprotobuf-c? It would really add a lot of bulk to the library as the linker will just discard it if it's not used. Or I could make a different target for it - say libprotobuf-c-text.

Thoughts?

I cloned protobuf-c from git master and trying to make it run but i Have following errors:

After configuring the output looks like this :

I am wondering whats wrong there.

While building protobuf-c using the provided cmake file I encountered the following linker error (the first of many similar errors):

libprotoc.lib(code_generator.cc.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '0' doesn't
match value '2' in c_bytes_field.obj [C:\Users\brian\Documents\protobuf-c\build-cmake\build\protoc-gen-c.vcxproj]
libprotoc.lib(code_generator.cc.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MT_StaticRelease'
doesn't match value 'MTd_StaticDebug' in c_bytes_field.obj [C:\Users\brian\Documents\protobuf-c\build-cmake\build\proto
c-gen-c.vcxproj]

Here are the steps I followed and the build output. I cloned protobuf-c from master.

C:\Users\brian\Documents\protobuf-c\build-cmake> mkdir build && cd build
C:\Users\brian\Documents\protobuf-c\build-cmake\build> cmake ..
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19045.
-- The C compiler identification is MSVC 19.31.31104.0
-- The CXX compiler identification is MSVC 19.31.31104.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
C:/Users/brian/Documents/protobuf-c
-- Found Protobuf: C:/Users/brian/Documents/protobuf/lib/libprotobuf.lib (found version "3.21.12")
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/brian/Documents/protobuf-c/build-cmake/build

C:\Users\brian\Documents\protobuf-c\build-cmake\build> cmake --build .

It's a pretty basic usage of cmake, so I'm assuming it's a bug. Compilation works fine but linking doesn't. Here's what else I tried:

cmake -DCMAKE_BUILD_TYPE=Release .. cmake -G "Visual Studio 15 2017 Win64" .. to select the Visual Studio 2017 tools.

now the proto convert to <xxxpb-c.h> has a macro like this : #define P_DETECT__IFIT_FLOWINFO__INIT \ { PROTOBUF_C_MESSAGE_INIT (&p_detect__ifit_flowinfo__descriptor) \ , NULL, 0, NULL, 0, 0, 0, 0, 0, 0, (char *)protobuf_c_empty_string, 0 } however i don't want the pointer filed initialized as NULL point, is there any solution?

Hi,

the current git repos does not compile under Debian 11.5

uname 
  5.10.0-19-amd64
g++ --version
  g++ (Debian 10.2.1-6) 10.2.1

git clone https://github.com/protobuf-c/protobuf-c.git
cd protobuf-c
./autogen.sh
./configure
    protobuf-c 1.4.1

        CC:                     gcc
        CFLAGS:                 -g -O2
        CXX:                    g++ -std=c++11
        CXXFLAGS:               -g -O2
        LDFLAGS:                
        LIBS:                   

        prefix:                 /usr/local
        sysconfdir:             ${prefix}/etc
        libdir:                 ${exec_prefix}/lib
        includedir:             ${prefix}/include
        pkgconfigdir:           ${libdir}/pkgconfig

        bigendian:              no
        protobuf version:       libprotoc 21.9
        
make
  GEN      protobuf-c/protobuf-c.pb.cc
  CXX      protoc-c/protoc_gen_c-c_bytes_field.o
In file included from ./protoc-c/c_field.h:67,
                 from ./protoc-c/c_bytes_field.h:68,
                 from protoc-c/c_bytes_field.cc:63:
/usr/local/include/google/protobuf/stubs/common.h:63:2: error: #error "C++ versions less than C++14 are not supported."
   63 | #error "C++ versions less than C++14 are not supported."
      |  ^~~~~
In file included from ./protoc-c/c_bytes_field.h:68,
                 from protoc-c/c_bytes_field.cc:63:
./protoc-c/c_field.h:108:3: error: ISO C++ forbids declaration of ‘GOOGLE_DISALLOW_EVIL_CONSTRUCTORS’ with no type [-fpermissive]
  108 |   GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(FieldGenerator);
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
....

Editing Makefile and change to

CXX = g++ -std=c++14
CXXCPP = g++ -E -std=c++14

does not help:

make
  CXX      protoc-c/protoc_gen_c-c_bytes_field.o
In file included from ./protoc-c/c_bytes_field.h:68,
                 from protoc-c/c_bytes_field.cc:63:
./protoc-c/c_field.h:108:3: error: ISO C++ forbids declaration of ‘GOOGLE_DISALLOW_EVIL_CONSTRUCTORS’ with no type [-fpermissive]
  108 |   GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(FieldGenerator);
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./protoc-c/c_field.h:108:37: error: cannot declare parameter to be of abstract type ‘google::protobuf::compiler::c::FieldGenerator’
  108 |   GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(FieldGenerator);
....

How to resolve?

Thanks Marcel

Add CIFuzz workflow action to have fuzzers build and run on each PR.

This is a service offered by OSS-Fuzz where protobuf-c already runs (https://github.com/google/oss-fuzz/tree/master/projects/protobuf-c). CIFuzz can help detect regressions and catch fuzzing build issues early, and has a variety of features (see the URL above). In the current PR the fuzzers gets build on a pull request and will run for 120 seconds.

Hi,

Maybe a silly question: but which C (not C++) grpc library do you use with protobuf-c?

(I searched a bit, but didn't find any good / active maintained C (not C++) grpc library on the web).

Or may I use a Rust (or some other language's) grpc library with protobuf-c? has anyone tried that? any tutorials?

Thanks.