Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update protobuf3 and use CMake #561

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

dstoup
Copy link
Collaborator

@dstoup dstoup commented Oct 28, 2019

Currently this branch sets the default Protobuf version to 3 for testing purposes only. We have more work to do before version 3 can be the default.l If this branch passes dashboards, I will pull the version back to 2 and repush.

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@dstoup dstoup force-pushed the cmake-upgrade-protobuf3 branch from e31efc8 to 80fcb9e Compare October 30, 2019 13:30
@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@dstoup
Copy link
Collaborator Author

dstoup commented Oct 30, 2019

Jenkins test this please

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@dstoup
Copy link
Collaborator Author

dstoup commented Oct 31, 2019

@aaron-bray The dashboards passed but I think you'd mentioned something about kwiver's definitions needing to change as a result of pb3. Should I keep 2 as the default for now? Thinking that's the right thing anyway until we get buyin, but wanted to get a sense of the rest of the work left to do.

${COMMON_EP_ARGS}
${COMMON_CMAKE_EP_ARGS}
# UPDATE_COMMAND
# COMMAND ${CMAKE_COMMAND} -DPULSE_IL2CPP_PATCH=${PULSE_IL2CPP_PATCH} -Dprotobuf_source=${protobuf_SRC} -Dprotobuf_patch=${protobuf_Patch} -P ${protobuf_Patch}/Patch.cmake
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can take this patch out. It was put in to work with the Unity IL2CPP compiler that converts C# to C++.
The IL2CPP compiler can be a bit too aggressive, so this patch was created
By the time you are supporting kwiver/fletch into C# and Unity, this will probably have changed
Just let me know then

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks ... what are you thoughts about making protobuf3 the default? It certainly works building/testing kwiver on all platforms.

${COMMON_CMAKE_ARGS}
-Dprotobuf_BUILD_TESTS:BOOL=OFF
-Dprotobuf_BUILD_EXAMPLES:BOOL=OFF
-Dprotobuf_BUILD_SHARED_LIBS:BOOL=${BUILD_SHARED_LIBS}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please read this tid bit about protobuf linkage
https://github.com/protocolbuffers/protobuf/tree/master/cmake#dlls-vs-static-linking

I suggest we always build protobuf as static libs, and they should only be linked into the protobuf arrow and nothing else. Should work great in the kwiver architecture.

@aaron-bray
Copy link
Member

protobuf3 can read in and use protobuf2 files
I have also built protobuf3 in fletch and used that in a Caffe build, which has protobuf2 proto files, and everything still worked fine.

So you really don't need to move the kwiver proto files fro 2 to 3 immediately
It does not look like protobuf2 supports JSON
And there are other differences between the versions
https://groups.google.com/forum/#!topic/protobuf/h-nwPLb42ac

You should be able to remove protobuf2 altogether from fletch and just use 3
I think we should move to protobuf3 proto files unless there is a compelling reason to stay at 2
But we can do that on our own time and still only build protobuf3

So I feel pretty confident we can nix proto2 from fletch all together and not sink any ships

@dstoup
Copy link
Collaborator Author

dstoup commented Nov 15, 2019

Perfect, thanks! I will keep protobuf3 as the default, leave protobuf2 for just a little while and look to remove it completely soon. I'm looking into the static thing as well. I suspect I will have to force PIC in those cases since it didn't build correctly out of the box that way (on Linux). I think VS was fine building static

@aaron-bray
Copy link
Member

Note,
I think using the protobuf3 library (even with protobuf2 proto files) we should still get JSON support...
But I would want to test that just to be sure...

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@dstoup dstoup force-pushed the cmake-upgrade-protobuf3 branch from a69052e to cafb37e Compare November 18, 2019 19:59
@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

@kwcvrobot
Copy link
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants