Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add scarb verify #1922

Merged
merged 90 commits into from
Feb 3, 2025
Merged
Show file tree
Hide file tree
Changes from 88 commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
c658194
feat: executable command
FroyaTheHen Jan 8, 2025
af5414d
Initial `cairo-prove`
DelevoXDG Jan 17, 2025
ccc3aa3
Support prover config; Restructure args
DelevoXDG Jan 17, 2025
972acdd
Restructure `cairo-execute` output directories
DelevoXDG Jan 17, 2025
6036329
Refactor; Improve error messages; Improve args
DelevoXDG Jan 17, 2025
0a404e4
Add tests
DelevoXDG Jan 17, 2025
55525ed
Refactor
DelevoXDG Jan 17, 2025
0da153d
Format
DelevoXDG Jan 17, 2025
cb4b7fb
Add `--execute` flag to run execution before `prove`
DelevoXDG Jan 18, 2025
3946387
Refactor: add helper functions; Simplify `execution_number` extractio…
DelevoXDG Jan 18, 2025
951e3d7
Improve error messages in line with `scarb execute` changes
DelevoXDG Jan 18, 2025
ff83e91
use lossy utf8 conversion
DelevoXDG Jan 18, 2025
3c31623
misc: rename test
DelevoXDG Jan 18, 2025
5bb3424
rename all "execution number" mentions to `execution_id`
DelevoXDG Jan 18, 2025
01070f1
Add verbosity settings
DelevoXDG Jan 18, 2025
e02f52d
fix tests (`--execution-id`)
DelevoXDG Jan 18, 2025
e82596a
Disable `scarb execute` output for quiet verbosity
DelevoXDG Jan 18, 2025
39f1f70
Stream AND write output of subcommand on the fly
DelevoXDG Jan 18, 2025
de0d76d
Format
DelevoXDG Jan 18, 2025
72e223b
improve approach to streaming stdout of nested commands
DelevoXDG Jan 18, 2025
2ab30cd
refactor: rename scarb-cairo-{execute,prove} to scarb-{execute,prove}
DelevoXDG Jan 18, 2025
e6556de
finish `scarb-cairo-{execute,prove}` -> `scarb-{execute,prove}``
DelevoXDG Jan 18, 2025
c7e0dc6
Merge branch 'main' into zdobnikau/stwo-prove
DelevoXDG Jan 18, 2025
242e179
format
DelevoXDG Jan 18, 2025
48a1399
lock
DelevoXDG Jan 18, 2025
375195c
update ci
DelevoXDG Jan 20, 2025
a9e7e28
fix `scarb execute` tests
DelevoXDG Jan 21, 2025
d19a6e1
`--exclude` not supported by rustfmt
DelevoXDG Jan 21, 2025
03593e3
install components in `check-rust-scarb-prove`
DelevoXDG Jan 21, 2025
e9e2a15
Fix build in `test-scarb-prove`
DelevoXDG Jan 21, 2025
60de69f
fmt
DelevoXDG Jan 21, 2025
e42b960
fix `check-rust`
DelevoXDG Jan 21, 2025
eea251e
fix test-scarb-prove
DelevoXDG Jan 21, 2025
21a899c
fmt
DelevoXDG Jan 21, 2025
a1e1fe2
remove config.toml
DelevoXDG Jan 21, 2025
fa3ddf4
fix clippy ci
DelevoXDG Jan 21, 2025
6936618
fix tests
DelevoXDG Jan 21, 2025
a3ce3a4
fix ci
DelevoXDG Jan 21, 2025
d783d03
support arguments file
DelevoXDG Jan 21, 2025
a17eb8d
fmt
DelevoXDG Jan 21, 2025
57839d4
fmt
DelevoXDG Jan 21, 2025
1eb0f34
fmt
DelevoXDG Jan 21, 2025
4218467
update `extract_execution_id`
DelevoXDG Jan 21, 2025
67fb1ac
Link #1915
DelevoXDG Jan 24, 2025
78619fe
add warning re proof instability
DelevoXDG Jan 24, 2025
e3bc01b
fix test
DelevoXDG Jan 24, 2025
b435a2e
misc
DelevoXDG Jan 24, 2025
3e4cdd6
misc
DelevoXDG Jan 27, 2025
396bdff
add scarb-verify
DelevoXDG Jan 27, 2025
e337c55
update ci
DelevoXDG Jan 27, 2025
707a3ff
fix test
DelevoXDG Jan 27, 2025
679947f
format
DelevoXDG Jan 27, 2025
f87b63d
Revert "remove config.toml"
DelevoXDG Jan 28, 2025
4203a8c
add missing env: in ci
DelevoXDG Jan 28, 2025
063e8a3
ensure can format
DelevoXDG Jan 28, 2025
91a2e0c
add requested comment
DelevoXDG Jan 28, 2025
c8fe963
add missing env: in ci
DelevoXDG Jan 28, 2025
662b438
run `cargo fmt` inside a single job
DelevoXDG Jan 28, 2025
ce6c534
ci: remove unnecessary build step in `test-scarb-prove`
DelevoXDG Jan 28, 2025
1202a47
Merge branch 'main' into zdobnikau/stwo-prove
DelevoXDG Jan 28, 2025
e50f09c
lock
DelevoXDG Jan 28, 2025
7b2c5eb
remove support for path inputs
DelevoXDG Jan 28, 2025
8231ae5
require `--execute` as default arg
DelevoXDG Jan 28, 2025
8f6d6ca
fix order of "proving" msg
DelevoXDG Jan 28, 2025
a000eeb
split cairo-execute into subfiles
DelevoXDG Jan 28, 2025
a969fce
split scarb-execute into subfiles
DelevoXDG Jan 28, 2025
14c6a42
Merge branch 'zdobnikau/split-execute' into zdobnikau/stwo-prove
DelevoXDG Jan 28, 2025
cef4601
refactor `execute` args
DelevoXDG Jan 28, 2025
50482ed
update scarb-execute::main_inner to return execution_id
DelevoXDG Jan 28, 2025
cdac854
call scarb-execute directly in scarb-prove to retireve execution_id
DelevoXDG Jan 28, 2025
bd68867
ci: cache on nightly in `test-scarb-prove`
DelevoXDG Jan 28, 2025
e31ae82
remove unnecessary changes related to stdout execution_id extraction
DelevoXDG Jan 28, 2025
b2493e1
handle cairo-pie output explicitly
DelevoXDG Jan 28, 2025
6503636
use assert instead of ensure
DelevoXDG Jan 31, 2025
f23c968
apply suggestions re. `scarb-execute` re-use
DelevoXDG Jan 31, 2025
d6f9446
cairo-pie.zip
DelevoXDG Jan 31, 2025
1864fe2
cairo-pie.zip: refactor cleanup
DelevoXDG Jan 31, 2025
779d062
Merge branch 'main' into zdobnikau/stwo-prove
DelevoXDG Jan 31, 2025
21a2789
post-merge fixes
DelevoXDG Jan 31, 2025
3f678dd
drop windows support
DelevoXDG Jan 31, 2025
97e321e
revert thiserror dependency
DelevoXDG Jan 31, 2025
ce83714
relocate warning
DelevoXDG Jan 31, 2025
d7ead4f
Merge branch 'zdobnikau/stwo-prove' into zdobnikau/verify
DelevoXDG Jan 31, 2025
96b150b
minor CI fix
DelevoXDG Jan 31, 2025
e1f12b7
move stwo deps to workspace Cargo.toml
DelevoXDG Jan 31, 2025
f0d762b
fix tests
DelevoXDG Jan 31, 2025
8f9424d
disable windows tests
DelevoXDG Jan 31, 2025
4b8cd79
Merge branch 'main' into zdobnikau/verify
DelevoXDG Feb 3, 2025
608e0a4
ci: merge prove and verify testing steps
DelevoXDG Feb 3, 2025
486bb35
build
DelevoXDG Feb 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 13 additions & 11 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- uses: Swatinem/rust-cache@v2
- uses: taiki-e/install-action@nextest
- name: nextest archive
run: cargo nextest archive --workspace --all-features --cargo-profile ci --archive-file 'nextest-archive-${{ matrix.platform.os }}.tar.zst' --exclude scarb-prove
run: cargo nextest archive --workspace --all-features --cargo-profile ci --archive-file 'nextest-archive-${{ matrix.platform.os }}.tar.zst' --exclude scarb-prove --exclude scarb-verify
- uses: actions/upload-artifact@v4
with:
name: nextest-archive-${{ matrix.platform.os }}
Expand Down Expand Up @@ -82,8 +82,8 @@ jobs:
- name: run tests
run: cargo test -p scarb-metadata

test-scarb-prove:
name: test scarb-prove ${{ matrix.platform.name }}
test-nightly-crates:
name: test nightly crates ${{ matrix.platform.name }}
runs-on: ${{ matrix.platform.os }}
env:
# TODO(#1915): Use stable toolchain once stwo is stable.
Expand All @@ -104,9 +104,11 @@ jobs:
toolchain: ${{ env.RUST_TOOLCHAIN }}
- uses: Swatinem/rust-cache@v2
- name: Build
run: cargo +stable build --workspace --all-features --exclude scarb-prove
run: cargo +stable build --workspace --all-features --exclude scarb-prove --exclude scarb-verify
- name: Run scarb-prove tests
run: cargo +${{ env.RUST_TOOLCHAIN }} test -p scarb-prove
- name: Run scarb-verify tests
run: cargo +${{ env.RUST_TOOLCHAIN }} test -p scarb-verify
DelevoXDG marked this conversation as resolved.
Show resolved Hide resolved

test-prebuilt-plugins:
name: test prebuilt plugins ${{ matrix.platform.name }}
Expand Down Expand Up @@ -140,17 +142,17 @@ jobs:
- uses: Swatinem/rust-cache@v2
- run: cargo fmt --check
# TODO(#1915): Build all crates with stable toolchain once stwo is stable.
- run: cargo clippy --all-targets --all-features --workspace --exclude scarb-prove -- --no-deps
- run: cargo clippy --all-targets --all-features --workspace --exclude scarb-prove --exclude scarb-verify -- --no-deps
env:
# Make sure CI fails on all warnings, including Clippy lints.
RUSTFLAGS: "-Dwarnings"
- run: cargo doc --all-features --no-deps --workspace --exclude scarb-prove
- run: cargo doc --all-features --no-deps --workspace --exclude scarb-prove --exclude scarb-verify
env:
# Make sure CI fails on all warnings, including Clippy lints.
RUSTDOCFLAGS: "-Dwarnings"

check-rust-scarb-prove:
name: check-rust (scarb-prove)
check-rust-nightly:
name: check-rust (nightly)
runs-on: ubuntu-latest
env:
# TODO(#1915): Use stable toolchain once stwo is stable.
Expand All @@ -162,14 +164,14 @@ jobs:
toolchain: ${{ env.RUST_TOOLCHAIN }}
components: rustfmt, clippy
- uses: Swatinem/rust-cache@v2
- run: cargo +${{ env.RUST_TOOLCHAIN }} clippy --all-targets --all-features -p scarb-prove -- --no-deps
- run: cargo +${{ env.RUST_TOOLCHAIN }} clippy --all-targets --all-features -p scarb-prove -p scarb-verify -- --no-deps
env:
# Make sure CI fails on all warnings, including Clippy lints.
RUSTFLAGS: "-Dwarnings"
- run: cargo +${{ env.RUST_TOOLCHAIN }} doc --all-features --no-deps -p scarb-prove
- run: cargo +${{ env.RUST_TOOLCHAIN }} doc --all-features --no-deps -p scarb-prove -p scarb-verify
env:
# Make sure CI fails on all warnings, including Clippy lints.
RUSTDOCFLAGS: "-Dwarnings"
RUSTDOCFLAGS: "-Dwarnings"

check-website:
runs-on: ubuntu-latest
Expand Down
3 changes: 3 additions & 0 deletions Cargo.toml
DelevoXDG marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ members = [
"extensions/scarb-doc",
"extensions/scarb-execute",
"extensions/scarb-prove",
"extensions/scarb-verify",
"extensions/scarb-cairo-language-server",
"extensions/scarb-cairo-run",
"extensions/scarb-cairo-test",
Expand Down Expand Up @@ -76,6 +77,8 @@ cairo-lang-utils = { version = "*", features = ["env_logger"] }
cairo-language-server = "*"
cairo-lint-core = "*"
cairo-vm = "1.0.1"
stwo_cairo_prover = { git = "https://github.com/starkware-libs/stwo-cairo", rev = "a27287c21e8b7e677b07cc40988c8c145533c55d" }
stwo-prover = { git = "https://github.com/starkware-libs/stwo", rev = "af5475cb", features = ["parallel"] }
camino = { version = "1", features = ["serde1"] }
cargo_metadata = ">=0.18"
clap = { version = "4", features = ["derive", "env", "string"] }
Expand Down
7 changes: 2 additions & 5 deletions extensions/scarb-prove/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,12 @@ serde.workspace = true
camino.workspace = true
serde_json.workspace = true
create-output-dir = { path = "../../utils/create-output-dir" }
stwo_cairo_prover = { git = "https://github.com/starkware-libs/stwo-cairo", rev = "a27287c21e8b7e677b07cc40988c8c145533c55d" }
stwo-prover = { git = "https://github.com/starkware-libs/stwo", rev = "af5475cb", features = [
"parallel",
] }
stwo_cairo_prover.workspace = true
stwo-prover.workspace = true
scarb-execute = { path = "../scarb-execute" }

[dev-dependencies]
assert_fs.workspace = true
scarb-test-support = { path = "../../utils/scarb-test-support" }
snapbox.workspace = true
predicates.workspace = true

28 changes: 28 additions & 0 deletions extensions/scarb-verify/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
[package]
name = "scarb-verify"
publish = false

authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
readme.workspace = true
repository.workspace = true
version.workspace = true

[dependencies]
anyhow.workspace = true
indoc.workspace = true
scarb-metadata = { path = "../../scarb-metadata" }
scarb-ui = { path = "../../utils/scarb-ui" }
clap.workspace = true
serde.workspace = true
camino.workspace = true
serde_json.workspace = true
stwo_cairo_prover.workspace = true
stwo-prover.workspace = true

[dev-dependencies]
assert_fs.workspace = true
scarb-test-support = { path = "../../utils/scarb-test-support" }
snapbox.workspace = true
119 changes: 119 additions & 0 deletions extensions/scarb-verify/src/main.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
use anyhow::{ensure, Context, Result};
use camino::{Utf8Path, Utf8PathBuf};
use clap::Parser;
use indoc::formatdoc;
use scarb_metadata::{MetadataCommand, PackageMetadata};
use scarb_ui::args::{PackagesFilter, VerbositySpec};
use scarb_ui::components::Status;
use scarb_ui::{OutputFormat, Ui};
use std::env;
use std::fs;
use std::process::ExitCode;
use stwo_cairo_prover::cairo_air::air::CairoProof;
use stwo_cairo_prover::cairo_air::verify_cairo;
use stwo_prover::core::vcs::blake2_merkle::{Blake2sMerkleChannel, Blake2sMerkleHasher};

/// Verifies `scarb prove` output using Stwo verifier.
#[derive(Parser, Clone, Debug)]
#[clap(version, verbatim_doc_comment)]
struct Args {
/// Name of the package.
#[command(flatten)]
packages_filter: PackagesFilter,

/// ID of `scarb execute` output for given package, for which proof was generated using `scarb prove`.
#[arg(long)]
execution_id: Option<u32>,

/// Proof file path.
#[arg(
long,
required_unless_present = "execution_id",
conflicts_with = "execution_id"
)]
proof_file: Option<Utf8PathBuf>,

/// Logging verbosity.
#[command(flatten)]
pub verbose: VerbositySpec,
}

fn main() -> ExitCode {
let args = Args::parse();
let ui = Ui::new(args.verbose.clone().into(), OutputFormat::Text);

match main_inner(args, ui.clone()) {
Ok(()) => ExitCode::SUCCESS,
Err(error) => {
ui.error(format!("{error:#}"));
ExitCode::FAILURE
}
}
}

fn main_inner(args: Args, ui: Ui) -> Result<()> {
let scarb_target_dir = Utf8PathBuf::from(env::var("SCARB_TARGET_DIR")?);

let metadata = MetadataCommand::new().inherit_stderr().exec()?;
let package = args.packages_filter.match_one(&metadata)?;

let proof_path = if let Some(execution_id) = args.execution_id {
ui.print(Status::new("Verifying", &package.name));
resolve_proof_path_from_package(&scarb_target_dir, &package, execution_id)?
} else {
ui.print(Status::new("Verifying", "proof"));
args.proof_file.unwrap()
};

let proof = load_proof(&proof_path)?;

verify_cairo::<Blake2sMerkleChannel>(proof).with_context(|| "failed to verify proof")?;

ui.print(Status::new("Verified", "proof successfully"));

Ok(())
}

fn load_proof(path: &Utf8Path) -> Result<CairoProof<Blake2sMerkleHasher>> {
ensure!(
path.exists(),
format!("proof file does not exist at path: {path}")
);

let proof_contents =
fs::read_to_string(path).with_context(|| format!("failed to read proof file: {path}"))?;
let proof = serde_json::from_str(&proof_contents)
.with_context(|| format!("failed to deserialize proof file: {path}"))?;
Ok(proof)
}

fn resolve_proof_path_from_package(
scarb_target_dir: &Utf8Path,
package: &PackageMetadata,
execution_id: u32,
) -> Result<Utf8PathBuf> {
let execution_dir = scarb_target_dir
.join("execute")
.join(&package.name)
.join(format!("execution{execution_id}"));

ensure!(
execution_dir.exists(),
formatdoc! {r#"
execution directory does not exist at path: {execution_dir}
help: make sure to run `scarb prove --execute` first
and that the execution ID is correct
"#}
);

let proof_path = execution_dir.join("proof").join("proof.json");
ensure!(
proof_path.exists(),
formatdoc! {r#"
proof file does not exist at path: {proof_path}
help: run `scarb prove --execution-id={execution_id}` first
"#}
);

Ok(proof_path)
}
137 changes: 137 additions & 0 deletions extensions/scarb-verify/tests/build.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
use assert_fs::TempDir;
use indoc::indoc;
use scarb_test_support::command::Scarb;
use scarb_test_support::project_builder::ProjectBuilder;
use snapbox::cmd::OutputAssert;

fn build_executable_project() -> TempDir {
let t = TempDir::new().unwrap();
ProjectBuilder::start()
.name("hello")
.version("0.1.0")
.dep_cairo_execute()
.manifest_extra(indoc! {r#"
[executable]

[cairo]
enable-gas = false
"#})
.lib_cairo(indoc! {r#"
#[executable]
fn main() -> felt252 {
42
}
"#})
.build(&t);
t
}

// Disabled due to `scarb prove` not being supported on Windows
#[cfg(not(windows))]
#[test]
fn verify_from_execution_output() {
let t = build_executable_project();

Scarb::quick_snapbox()
.arg("execute")
.current_dir(&t)
.assert()
.success();

Scarb::quick_snapbox()
.arg("prove")
.arg("--execution-id=1")
.current_dir(&t)
.assert()
.success();

Scarb::quick_snapbox()
.arg("verify")
.arg("--execution-id=1")
.current_dir(&t)
.assert()
.success()
.stdout_matches(indoc! {r#"
[..]Verifying hello
[..]Verified proof successfully
"#});
}

// Disabled due to `scarb prove` not being supported on Windows
#[cfg(not(windows))]
#[test]
fn verify_from_path() {
let t = build_executable_project();

Scarb::quick_snapbox()
.arg("execute")
.current_dir(&t)
.assert()
.success();

Scarb::quick_snapbox()
.arg("prove")
.arg("--execution-id=1")
.current_dir(&t)
.assert()
.success();

Scarb::quick_snapbox()
.arg("verify")
.arg("--proof-file=target/execute/hello/execution1/proof/proof.json")
.current_dir(&t)
.assert()
.success()
.stdout_matches(indoc! {r#"
[..]Verifying proof
[..]Verified proof successfully
"#});
}

#[test]
fn verify_fails_when_execution_output_not_found() {
let t = build_executable_project();

output_assert(
Scarb::quick_snapbox()
.arg("verify")
.arg("--execution-id=1")
.current_dir(&t)
.assert()
.failure(),
indoc! {r#"
[..]Verifying hello
error: execution directory does not exist at path: [..]/target/execute/hello/execution1
help: make sure to run `scarb prove --execute` first
and that the execution ID is correct

"#},
)
}

#[test]
fn verify_fails_when_proof_file_not_found() {
let t = build_executable_project();

output_assert(
Scarb::quick_snapbox()
.arg("verify")
.arg("--proof-file=nonexistent.json")
.current_dir(&t)
.assert()
.failure(),
indoc! {r#"
[..]Verifying proof
error: proof file does not exist at path: nonexistent.json
"#},
)
}

fn output_assert(output: OutputAssert, expected: &str) {
#[cfg(windows)]
output.stdout_matches(format!(
"{expected}error: process did not exit successfully: exit code: 1\n"
));
#[cfg(not(windows))]
output.stdout_matches(expected);
}
Loading