refactor: make bitcoin implementation flexible #1092

Abdulkbk · 2024-12-30T20:45:25Z

Description

This PR refactors the bitcoin implementation.

Currently, the implementation only supports bitcoind, making it less flexible. This PR allows for the easy integration of other bitcoin implementations, such as BTCD, in the future.

Approach:
The PR refactors the Bitcoin implementation to utilize BitcoinFactory, similar to the implementation for the Lightning network, which allows for various Lightning implementations in Polar. This change will also improve the flexibility of the Bitcoin implementation.

Steps to Test

Since this is a refactor PR and does not change any behavior in Polar, you can test it normally by starting Polar and creating a network that involves bitcoind.

Here are a few pointers to test:

Conduct transactions between two or more bitcoind nodes.
Utilize bitcoind backends for some Lightning nodes and perform various operations.

codecov · 2024-12-30T20:51:40Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (f15f5c6) to head (703c234).
Report is 3 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##            master     #1092   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          152       154    +2     
  Lines         5547      5567   +20     
  Branches      1128      1129    +1     
=========================================
+ Hits          5547      5567   +20

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

jamaljsr · 2025-01-03T17:03:37Z

Hey @Abdulkbk thanks for opening this draft PR to show what you've implemented so far. I skimmed through the diff and so far it's looking fantastic! I'm pleasantly surprised to see the test coverage still at 100%. 👌

Please let me know if you have any questions or need any assistance moving forward with this.

Abdulkbk · 2025-01-06T09:10:43Z

Hey @jamaljsr thanks for taking the time to skim through this.

I'm pleasantly surprised to see the test coverage still at 100%.

Yes, I try to add test to all new code as I add them👌.

Please let me know if you have any questions or need any assistance moving forward with this.

Of course I will. I've been thinking of the best way I should add thebtcdservice but I'm inclined towards using REST as I couldn't find a standard library similar to the one used for bitcoindService.

Abdulkbk · 2025-01-15T18:34:53Z

Hey @jamaljsr, I am currently facing two blockers. I figured out a way for the first but I am still figuring out the second.

The first is that BTCD has wallet functionalities decoupled from chain functionalities, unlike bitcoind. However, there are workarounds. For example, I can run BTCD alongside btcwallet, and then use btcwallet for all non-CLI calls, as it supports both chain and wallet-related RPCs.

The second blocker is that btcwallet does not support regtest currently, meanwhile all the other node implementations in Polar use regtest. I think this is the real challenge.

I would love to hear your thoughts as I continue to figure a way out. Thanks.

jamaljsr · 2025-01-16T06:30:27Z

Hi @Abdulkbk,

The second blocker is that btcwallet does not support regtest currently, meanwhile all the other node implementations in Polar use regtest. I think this is the real challenge.

I would love to hear your thoughts as I continue to figure a way out. Thanks.

Oh wow, you are correct! btcd uses a different network for local testing called simnet. I totally forgot about this. While lnd does have support for simnet, the other implementations do not AFAIK. So if we move forward, we'd have to prevent adding CLN/Eclair nodes when there is no bitcoind present in the network. We'd also have to prevent mixing btcd and bitcoind in the same network since they won't be able to communicate with each other. This problem is going to really increase the effort required for this PR substantially due to the added compatibility complexity.

I honestly don't have all the answers right now on how to implement this best, but I'm willing to work with you if you are still interested in pursuing this. My hunch is that there will be more challenges along the way. We'll likely need to split this up into multiple PRs so reviewing the updates won't be too difficult. Let me know if you'd like to continue with this. I'd totally understand if this feels like too large of a project to tackle at this time.

Abdulkbk · 2025-01-16T11:54:12Z

This problem is going to really increase the effort required for this PR substantially due to the added compatibility complexity.

I agree this will require much more effort.

We'll likely need to split this up into multiple PRs so reviewing the updates won't be too difficult. Let me know if you'd like to continue with this. I'd totally understand if this feels like too large of a project to tackle at this time.

I would like to take this further. I will perhaps start by modifying this PR to focus on refactoring the implementation of the Bitcoin nodes to use a factory. Once I’ve completed that, I will request a review.

Abdulkbk · 2025-01-20T21:25:10Z

Hey @jamaljsr, I think this PR is ready for review. Thanks

jamaljsr

@Abdulkbk Thank you very much for the updates to this PR. This is very well done. I wouldn't have done it any different myself. I really appreciate this update. 🙏

This looks good to go with just one small interface to cleanup. I'll be happy to merge this after that change and a rebase on master.

src/types/index.ts

In this commit, we refactor the designer components to make it possible for us to introduce another bitcoin node implementation later.

Abdulkbk · 2025-01-27T14:21:22Z

src/lib/bitcoin/bitcoind/bitcoindService.ts

  // the types in the v5 release are missing many functions so we have to cast as `any`
  // to prevent TS errors.
  createClient(node: BitcoinNode): any {
    return new BitcoinCore({
-      host: `http://127.0.0.1:${node.ports.rpc}`,
+      host: `127.0.0.1`,


I want also to inform you @jamaljsr that bitcoind could not connect after this change until I made this modification.

Before:

return new BitcoinCore({ host: `http://127.0.0.1:${node.ports.rpc}`, username: bitcoinCredentials.user, password: bitcoinCredentials.pass, logger: this.log(), // use a long timeout due to the time it takes to mine a lot of blocks timeout: 5 * 60 * 1000, });

After:

return new BitcoinCore({ host: `127.0.0.1`, port: node.ports.rpc, username: bitcoinCredentials.user, password: bitcoinCredentials.pass, logger: this.log(), // use a long timeout due to the time it takes to mine a lot of blocks timeout: 5 * 60 * 1000, });

Yes, the bitcoin-core package was updated to v5 by the bot in #1115. This was a breaking change that I fixed in master by modifying the constructor args. You'll need to run yarn install to ensure you have the latest package version after the rebase, which should fix the connection error without a code change.

Yes, the bitcoin-core package was updated to v5 by the bot in #1115. This was a breaking change that I fixed in master by modifying the constructor args. You'll need to run yarn install to ensure you have the latest package version after the rebase, which should fix the connection error without a code change.

I just ran yarn install after the rebase and the connection worked without the modification. I have removed this modification now.

In this commit, we refactor how bitcoind is implemented and used, and moved to using injected factory so that other node implementa tions can be added later.

In this commit, we refactor the test files to use the new bitcoin factory implementation. We also added a test for the newly created notImplemented service file for bitcoin nodes.

jamaljsr

LGTM 💪

Abdulkbk changed the title ~~Btcd support~~ Feat: btcd support Dec 30, 2024

Abdulkbk force-pushed the btcd-support branch from 17f4abb to ea5950e Compare January 16, 2025 12:41

Abdulkbk changed the title ~~Feat: btcd support~~ refactor: make bitcoin implementation flexible Jan 16, 2025

Abdulkbk force-pushed the btcd-support branch from ea5950e to 7cfbed7 Compare January 16, 2025 13:13

Abdulkbk marked this pull request as ready for review January 20, 2025 21:24

jamaljsr requested changes Jan 27, 2025

View reviewed changes

src/types/index.ts Show resolved Hide resolved

Abdulkbk force-pushed the btcd-support branch from 7cfbed7 to 5ca51f8 Compare January 27, 2025 08:35

refactor: make designer flexible for other nodes

a410433

In this commit, we refactor the designer components to make it possible for us to introduce another bitcoin node implementation later.

Abdulkbk force-pushed the btcd-support branch from 5ca51f8 to 49c1e3d Compare January 27, 2025 13:48

Abdulkbk commented Jan 27, 2025

View reviewed changes

Abdulkbk requested a review from jamaljsr January 27, 2025 14:21

Abdulkbk added 2 commits January 27, 2025 20:26

refactor: make use of factory for bitcoin nodes

b130983

In this commit, we refactor how bitcoind is implemented and used, and moved to using injected factory so that other node implementa tions can be added later.

refactor: use the new bitcoin factory in tests

703c234

In this commit, we refactor the test files to use the new bitcoin factory implementation. We also added a test for the newly created notImplemented service file for bitcoin nodes.

Abdulkbk force-pushed the btcd-support branch from 49c1e3d to 703c234 Compare January 27, 2025 19:30

jamaljsr approved these changes Jan 27, 2025

View reviewed changes

jamaljsr merged commit e7f31b8 into jamaljsr:master Jan 27, 2025
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: make bitcoin implementation flexible #1092

refactor: make bitcoin implementation flexible #1092

Abdulkbk commented Dec 30, 2024 •

edited

Loading

codecov bot commented Dec 30, 2024 •

edited

Loading

jamaljsr commented Jan 3, 2025

Abdulkbk commented Jan 6, 2025

Abdulkbk commented Jan 15, 2025

jamaljsr commented Jan 16, 2025

Abdulkbk commented Jan 16, 2025

Abdulkbk commented Jan 20, 2025 •

edited

Loading

jamaljsr left a comment •

edited

Loading

Abdulkbk Jan 27, 2025

jamaljsr Jan 27, 2025 •

edited

Loading

Abdulkbk Jan 27, 2025

jamaljsr left a comment

refactor: make bitcoin implementation flexible #1092

refactor: make bitcoin implementation flexible #1092

Conversation

Abdulkbk commented Dec 30, 2024 • edited Loading

Description

Steps to Test

codecov bot commented Dec 30, 2024 • edited Loading

Codecov Report

jamaljsr commented Jan 3, 2025

Abdulkbk commented Jan 6, 2025

Abdulkbk commented Jan 15, 2025

jamaljsr commented Jan 16, 2025

Abdulkbk commented Jan 16, 2025

Abdulkbk commented Jan 20, 2025 • edited Loading

jamaljsr left a comment • edited Loading

Choose a reason for hiding this comment

Abdulkbk Jan 27, 2025

Choose a reason for hiding this comment

jamaljsr Jan 27, 2025 • edited Loading

Choose a reason for hiding this comment

Abdulkbk Jan 27, 2025

Choose a reason for hiding this comment

jamaljsr left a comment

Choose a reason for hiding this comment

Abdulkbk commented Dec 30, 2024 •

edited

Loading

codecov bot commented Dec 30, 2024 •

edited

Loading

Abdulkbk commented Jan 20, 2025 •

edited

Loading

jamaljsr left a comment •

edited

Loading

jamaljsr Jan 27, 2025 •

edited

Loading