BUGFIX: Add support for nested definitions in Python mcap-protobuf-support #1321
+99
−2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
methylDragon
requested review from
jtbandes,
james-rms and
indirectlylit
as code owners
methylDragon
force-pushed
the
ch3/nested-proto-support
branch
2 times, most recently
from
a90cda9 to
7943adb
Compare
Signed-off-by: methylDragon <methylDragon@gmail.com>
methylDragon
force-pushed
the
ch3/nested-proto-support
branch
from
7943adb to
e1f655a
Compare
copybara-service bot
pushed a commit
to protocolbuffers/protobuf
that referenced
this pull request
Jan 26, 2025
Otherwise downstream callers must explicitly fetch and generate message classes for nested definitions. (For example [here](foxglove/mcap#1321).) Example: ``` message NestedTypeMessage { message NestedType { string data = 1; } NestedType nested = 1; } ``` Currently `GetMessageClassesForFiles()` will only return `NestedTypeMessage`, but miss `NestedTypeMessage.NestedType` (and any arbitrary level of nested definitions). This is an unexpected deviation from the comment that says that it will find and resolve all dependencies. PiperOrigin-RevId: 719854337
copybara-service bot
pushed a commit
to protocolbuffers/protobuf
that referenced
this pull request
Jan 26, 2025
Otherwise downstream callers must explicitly fetch and generate message classes for nested definitions. (For example [here](foxglove/mcap#1321).) Example: ``` message NestedTypeMessage { message NestedType { string data = 1; } NestedType nested = 1; } ``` Currently `GetMessageClassesForFiles()` will only return `NestedTypeMessage`, but miss `NestedTypeMessage.NestedType` (and any arbitrary level of nested definitions). This is an unexpected deviation from the comment that says that it will find and resolve all dependencies. PiperOrigin-RevId: 719854337
copybara-service bot
pushed a commit
to protocolbuffers/protobuf
that referenced
this pull request
Jan 26, 2025
Otherwise downstream callers must explicitly fetch and generate message classes for nested definitions. (For example [here](foxglove/mcap#1321).) Example: ``` message NestedTypeMessage { message NestedType { string data = 1; } NestedType nested = 1; } ``` Currently `GetMessageClassesForFiles()` will only return `NestedTypeMessage`, but miss `NestedTypeMessage.NestedType` (and any arbitrary level of nested definitions). This is an unexpected deviation from the comment that says that it will find and resolve all dependencies. PiperOrigin-RevId: 719854337
methylDragon
changed the title
james-rms
reviewed
Jan 27, 2025
| ) | ||
| while nested_descriptions: | ||
| nested_desc = nested_descriptions.pop() | ||
| nested_messages[nested_desc.full_name] = GetMessageClass( |
There was a problem hiding this comment.
why does this not mutate messages directly?
james-rms
approved these changes
Jan 27, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changelog
Bugfix: Support nested message definitions in mcap-protobuf-support
Description
Consider the following Protobuf definition:
Currently the decoder's use of the
protobuf.GetMessageClassesForFiles()misses out the innerNestedTypedefinition, causing decoding failures from missing nested descriptors, even though those descriptors would have been populated as part of the encoded file descriptors in an MCAP file.The error that is raised is something like:
FileDescriptorSet for type {schema.name} is missing that schemaThis issue persists to an arbitrary level of nesting.
This PR fixes that issue (including for deeply nested definitions) and adds a unit test to prevent regression.
PS:
How soon after merging this will it make it to PyPI?