Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cluster state types #2311

Closed
wants to merge 2 commits into from
Closed

Cluster state types #2311

wants to merge 2 commits into from

Conversation

muhamadazmy
Copy link
Contributor

@muhamadazmy muhamadazmy commented Nov 18, 2024

Cluster state types

Summary:
Types used by nodes to share cluster state


Stack created with Sapling. Best reviewed with ReviewStack.

@tillrohrmann tillrohrmann self-requested a review November 18, 2024 13:55
@muhamadazmy muhamadazmy force-pushed the pr2311 branch 3 times, most recently from 345539a to 61fe276 Compare November 19, 2024 10:28
@AhmedSoliman AhmedSoliman self-requested a review November 19, 2024 10:33
@muhamadazmy muhamadazmy force-pushed the pr2311 branch 2 times, most recently from 7f721f2 to 190361d Compare November 19, 2024 10:48
@muhamadazmy muhamadazmy changed the title Node state gossip Cluster state types Dec 16, 2024
@muhamadazmy muhamadazmy force-pushed the pr2311 branch 5 times, most recently from 1f17889 to 7def569 Compare December 17, 2024 12:05
@muhamadazmy muhamadazmy force-pushed the pr2311 branch 3 times, most recently from 46f4206 to 7837086 Compare December 17, 2024 17:48
#[serde_as]
#[derive(Debug, Clone, Eq, PartialEq, Serialize, Deserialize)]
pub struct NodeData {
pub timestamp: MillisSinceEpoch,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does timestamp refer to here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That should be last_update_time. I will rename it

crates/types/src/net/cluster_state.rs Outdated Show resolved Hide resolved
crates/types/src/net/cluster_state.rs Outdated Show resolved Hide resolved
crates/types/src/net/cluster_state.rs Outdated Show resolved Hide resolved
@muhamadazmy muhamadazmy force-pushed the pr2311 branch 5 times, most recently from cb5630b to b3973f4 Compare December 20, 2024 14:18
Summary:
derprecate old cluster state that included information about
partition state.

A new ClusterState object is introduced that only have livenss information
Summary:
Types used by nodes to share cluster state
Comment on lines 19 to 34
super::define_rpc! {
@request=GetNodeState,
@response=NodeStateResponse,
@request_target=TargetName::NodeGetNodeStateRequest,
@response_target=TargetName::NodeGetNodeStateResponse,
@request=GetPartitionsProcessorsState,
@response=PartitionsProcessorsStateResponse,
@request_target=TargetName::NodeGetPartitionsProcessorsStateRequest,
@response_target=TargetName::NodeGetPartitionsProcessorsStateResponse,
}

#[derive(Debug, Clone, Copy, Default, Serialize, Deserialize)]
pub struct GetNodeState {}
pub struct GetPartitionsProcessorsState {}

#[serde_as]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct NodeStateResponse {
pub struct PartitionsProcessorsStateResponse {
/// State of paritions processor per parition. Is set to None if this node is not a `Worker` node
#[serde_as(as = "Option<serde_with::Seq<(_, _)>>")]
pub partition_processor_state: Option<BTreeMap<PartitionId, PartitionProcessorStatus>>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

possibly we should rename the module to be src/net/worker.rs since this worker-specific messaging and not node-level anymore.

@muhamadazmy muhamadazmy deleted the pr2311 branch December 26, 2024 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants