Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

slow start-up when not at AWS Seattle offices (I am in Europe) #421

Open
mikn opened this issue Dec 19, 2024 · 2 comments
Open

slow start-up when not at AWS Seattle offices (I am in Europe) #421

mikn opened this issue Dec 19, 2024 · 2 comments

Comments

@mikn
Copy link

mikn commented Dec 19, 2024 •
edited
Loading

Hi! As mentioned during the community meeting, it takes me somewhere around 15-30 seconds before I get to the build stage. This is at a high level what happens:

Combined Application and Network Timeline

Time Type Event/Message
21:32:35 App Command started: strace -t -f -e trace=network twoliter build variant metal-dev
21:32:35 Net Socket: Initial connection to ECR (44.216.203.94:443)
21:32:35 Log INFO twoliter::project::lock - Resolving project references to check against lock file
21:32:35 Log INFO twoliter::project::lock::image - Resolving dependency image dependency 'bottlerocket-core-kit-5.0.0@public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.0.0'
21:32:35 Net Socket: Secondary ECR connection (44.216.203.94:443)
21:32:36 Net Socket: AWS service connection (75.2.101.78:443)
21:32:36 Net Socket: AWS service connection (99.83.145.10:443)
21:32:37 Net Socket: Additional ECR connection (44.216.203.94:443)
21:32:37 Net DNS: Failed NSCD lookup
21:32:41 Net IPv6: Multiple connection attempts failed (ENETUNREACH)
21:32:41 Log INFO twoliter::project::lock::image - Resolving dependency image dependency 'bottlerocket-kernel-kit-1.0.0@public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.0'
21:32:42 Net Socket: Additional AWS service connection (99.83.145.10:443)
21:32:43 Net Socket: AWS service connection (75.2.101.78:443)
21:32:44 Net Socket: Additional AWS service connection (99.83.145.10:443)
21:32:45 Net Socket: Additional ECR connection (44.216.203.94:443)
21:32:46 Net Socket: AWS service connection (75.2.101.78:443)
21:32:47 Net Socket: AWS service connection (99.83.145.10:443)
21:32:48 Net Socket: AWS service connection (3.164.226.167:443)
21:32:48 Log INFO twoliter::project::lock::image - Resolving dependency image dependency 'bottlerocket-sdk-0.50.0@public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0'
21:32:48 Net IPv6: Multiple failed connection attempts
21:32:49 Net Socket: Final ECR connection (44.216.203.94:443)
21:32:50 Net Socket: Connection termination begins
21:32:52 Log INFO cargo-make 0.37.23
21:32:52 Log WARN Unable to parse Cargo.toml via cargo-metadata, fallbacking
21:32:53 Log INFO Build File: /home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools/Makefile.toml
21:32:53 Log INFO Task: build
21:32:53 Log INFO Profile: development
21:32:53 Log INFO Running Task: setup
21:32:53 Log INFO Running Task: setup-build
21:32:53 Log INFO Running Task: fetch-sdk

I had this strace output analysed by an AI and produced this log - but this was of a broken run as you can see by the fact that Cargo.toml was not parse:able.

Perhaps we could at least do all of the images in parallel?
@cbgbt
Copy link
Contributor

cbgbt commented Dec 19, 2024

We originally did not do parallel image pulls because docker was the engine underlying the pulls, and we did not trust it to do them correctly.

But we don't have that constraint now. I'm not sure if it's the only thing to blame here, but we can look into that as a start.

@mikn
Copy link
Author

mikn commented Jan 24, 2025

Here is perhaps a slightly more informative trace log output;

mikn@hal8999 ~/d/m/s/p/b/bottlerocket ((4677390c)) [2]> twoliter --log-level trace build kit platform-kit 
[2025-01-24T08:46:09Z TRACE twoliter::common] exec; cmd=Command { std: "docker" "version" "--format" "{{.Server.Version}}", kill_on_drop: false } quiet=true
[2025-01-24T08:46:09Z DEBUG twoliter::common] Running: Command { std: "docker" "version" "--format" "{{.Server.Version}}", kill_on_drop: false }
[2025-01-24T08:46:09Z TRACE twoliter::project] load_or_find_project; user_path=None
[2025-01-24T08:46:09Z TRACE twoliter::project] Looking for Twoliter.toml in '.'
[2025-01-24T08:46:09Z TRACE twoliter::common::fs] canonicalize; path=/home/mikn/devel/molnett/src/platform/base/bottlerocket/Twoliter.toml
[2025-01-24T08:46:09Z TRACE twoliter::common::fs] read_to_string; path=/home/mikn/devel/molnett/src/platform/base/bottlerocket/Twoliter.toml
[2025-01-24T08:46:09Z TRACE twoliter::project] This project does not have a Release.toml file (this is not a problem)
[2025-01-24T08:46:09Z DEBUG twoliter::project::lock::verification] Cleaning up any existing tag files for resolved artifacts
[2025-01-24T08:46:09Z DEBUG twoliter::project::lock::verification] Removing existing verification tag file '/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/external-kits/.sdk-verified'
[2025-01-24T08:46:09Z DEBUG twoliter::project::lock::verification] Removing existing verification tag file '/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/external-kits/.kits-verified'
[2025-01-24T08:46:09Z DEBUG twoliter::project] Project file loaded from '/home/mikn/devel/molnett/src/platform/base/bottlerocket/Twoliter.toml'
[2025-01-24T08:46:09Z DEBUG twoliter::project::lock::verification] Cleaning up any existing tag files for resolved artifacts
[2025-01-24T08:46:09Z INFO  twoliter::project::lock] Resolving project references to check against lock file
[2025-01-24T08:46:09Z DEBUG twoliter::project::lock] Loading existing lockfile '/home/mikn/devel/molnett/src/platform/base/bottlerocket/Twoliter.lock'
[2025-01-24T08:46:09Z DEBUG twoliter::project::lock] Resolving kit 'bottlerocket-core-kit' image=bottlerocket-core-kit-5.3.0@public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0
[2025-01-24T08:46:09Z TRACE twoliter::project::lock::image] resolve; self=ImageResolver { image: ProjectImage { image: Image { name: ValidIdentifier("bottlerocket-core-kit"), version: Version { major: 5, minor: 3, patch: 0 }, vendor: ValidIdentifier("bottlerocket") }, vendor: Verbatim(VerbatimVendor { vendor_name: ValidIdentifier("bottlerocket"), vendor: Vendor { registry: "public.ecr.aws/bottlerocket" } }) }, skip_metadata_retrieval: false } image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } } image=bottlerocket-core-kit-5.3.0@public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0 uri=public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0
[2025-01-24T08:46:09Z INFO  twoliter::project::lock::image] Resolving dependency image dependency 'bottlerocket-core-kit-5.3.0@public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0'.
[2025-01-24T08:46:09Z TRACE twoliter::project::lock::image] get_manifest; self=ImageResolver { image: ProjectImage { image: Image { name: ValidIdentifier("bottlerocket-core-kit"), version: Version { major: 5, minor: 3, patch: 0 }, vendor: ValidIdentifier("bottlerocket") }, vendor: Verbatim(VerbatimVendor { vendor_name: ValidIdentifier("bottlerocket"), vendor: Vendor { registry: "public.ecr.aws/bottlerocket" } }) }, skip_metadata_retrieval: false } image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } } image=bottlerocket-core-kit-5.3.0@public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0 uri=public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0
[2025-01-24T08:46:09Z DEBUG twoliter::project::lock::image] Fetching image manifest. image=bottlerocket-core-kit-5.3.0@public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0 uri="public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0"
[2025-01-24T08:46:11Z TRACE twoliter::project::lock::image] calculate_digest; self=ImageResolver { image: ProjectImage { image: Image { name: ValidIdentifier("bottlerocket-core-kit"), version: Version { major: 5, minor: 3, patch: 0 }, vendor: ValidIdentifier("bottlerocket") }, vendor: Verbatim(VerbatimVendor { vendor_name: ValidIdentifier("bottlerocket"), vendor: Vendor { registry: "public.ecr.aws/bottlerocket" } }) }, skip_metadata_retrieval: false } image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } } image=bottlerocket-core-kit-5.3.0@public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0 uri=public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0
[2025-01-24T08:46:12Z DEBUG twoliter::project::lock::image] Calculated digest for locked image 'public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0': 'U6fyce19tV0QNvUux86If1eEQY8k/7M62sEsGZCI7Wc='
[2025-01-24T08:46:12Z DEBUG twoliter::project::lock::image] Extracting kit metadata from OCI image
[2025-01-24T08:46:12Z TRACE twoliter::project::lock::image] try_from_image; image_uri="public.ecr.aws/bottlerocket/bottlerocket-core-kit@sha256:6382bb77362d90bd08657e5e86f0a545d6c37955aeea27df08fdf5f1803a47ce" image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } }
[2025-01-24T08:46:12Z TRACE twoliter::project::lock::image] Extracting kit metadata from OCI image config image_uri="public.ecr.aws/bottlerocket/bottlerocket-core-kit@sha256:6382bb77362d90bd08657e5e86f0a545d6c37955aeea27df08fdf5f1803a47ce"
[2025-01-24T08:46:14Z TRACE twoliter::project::lock::image] Kit metadata retrieved from image config image_uri="public.ecr.aws/bottlerocket/bottlerocket-core-kit@sha256:6382bb77362d90bd08657e5e86f0a545d6c37955aeea27df08fdf5f1803a47ce" image_config=ConfigView { labels: {"dev.bottlerocket.kit.v2": "eyJraXQiOltdLCJuYW1lIjoiYm90dGxlcm9ja2V0LWNvcmUta2l0Iiwic2RrIjp7ImRpZ2VzdCI6IlJqcHkvZ1ZnQmhVL0I2OTZ4YUsxWTQvZHJ6NHBOSnUrZnl5WlNJazlvTEU9IiwibmFtZSI6ImJvdHRsZXJvY2tldC1zZGsiLCJzb3VyY2UiOiJwdWJsaWMuZWNyLmF3cy9ib3R0bGVyb2NrZXQvYm90dGxlcm9ja2V0LXNkazp2MC41MC4wIiwidmVuZG9yIjoiYm90dGxlcm9ja2V0IiwidmVyc2lvbiI6IjAuNTAuMCJ9LCJ2ZXJzaW9uIjoiNS4zLjAifQo="} } kit_metadata=<ImageMetadata(decoded) [ImageMetadata { name: "bottlerocket-core-kit", version: Version { major: 5, minor: 3, patch: 0 }, sdk: Image { name: ValidIdentifier("bottlerocket-sdk"), version: Version { major: 0, minor: 50, patch: 0 }, vendor: ValidIdentifier("bottlerocket") }, kits: [] }]>
[2025-01-24T08:46:14Z TRACE twoliter::project::lock::image] Checking that all manifests refer to the same kit.
[2025-01-24T08:46:14Z TRACE twoliter::project::lock::image] try_from_image; image_uri="public.ecr.aws/bottlerocket/bottlerocket-core-kit@sha256:ad3e66491771cfabfbb4364bba47ba08fb1b2d72642f3c53ac7c742b5a1b6e12" image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } }
[2025-01-24T08:46:14Z TRACE twoliter::project::lock::image] Extracting kit metadata from OCI image config image_uri="public.ecr.aws/bottlerocket/bottlerocket-core-kit@sha256:ad3e66491771cfabfbb4364bba47ba08fb1b2d72642f3c53ac7c742b5a1b6e12"
[2025-01-24T08:46:15Z TRACE twoliter::project::lock::image] Kit metadata retrieved from image config image_uri="public.ecr.aws/bottlerocket/bottlerocket-core-kit@sha256:ad3e66491771cfabfbb4364bba47ba08fb1b2d72642f3c53ac7c742b5a1b6e12" image_config=ConfigView { labels: {"dev.bottlerocket.kit.v2": "eyJraXQiOltdLCJuYW1lIjoiYm90dGxlcm9ja2V0LWNvcmUta2l0Iiwic2RrIjp7ImRpZ2VzdCI6IlJqcHkvZ1ZnQmhVL0I2OTZ4YUsxWTQvZHJ6NHBOSnUrZnl5WlNJazlvTEU9IiwibmFtZSI6ImJvdHRsZXJvY2tldC1zZGsiLCJzb3VyY2UiOiJwdWJsaWMuZWNyLmF3cy9ib3R0bGVyb2NrZXQvYm90dGxlcm9ja2V0LXNkazp2MC41MC4wIiwidmVuZG9yIjoiYm90dGxlcm9ja2V0IiwidmVyc2lvbiI6IjAuNTAuMCJ9LCJ2ZXJzaW9uIjoiNS4zLjAifQo="} } kit_metadata=<ImageMetadata(decoded) [ImageMetadata { name: "bottlerocket-core-kit", version: Version { major: 5, minor: 3, patch: 0 }, sdk: Image { name: ValidIdentifier("bottlerocket-sdk"), version: Version { major: 0, minor: 50, patch: 0 }, vendor: ValidIdentifier("bottlerocket") }, kits: [] }]>
[2025-01-24T08:46:15Z DEBUG twoliter::project::lock] Resolving kit 'bottlerocket-kernel-kit' image=bottlerocket-kernel-kit-1.0.2@public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2
[2025-01-24T08:46:15Z TRACE twoliter::project::lock::image] resolve; self=ImageResolver { image: ProjectImage { image: Image { name: ValidIdentifier("bottlerocket-kernel-kit"), version: Version { major: 1, minor: 0, patch: 2 }, vendor: ValidIdentifier("bottlerocket") }, vendor: Verbatim(VerbatimVendor { vendor_name: ValidIdentifier("bottlerocket"), vendor: Vendor { registry: "public.ecr.aws/bottlerocket" } }) }, skip_metadata_retrieval: false } image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } } image=bottlerocket-kernel-kit-1.0.2@public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2 uri=public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2
[2025-01-24T08:46:15Z INFO  twoliter::project::lock::image] Resolving dependency image dependency 'bottlerocket-kernel-kit-1.0.2@public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2'.
[2025-01-24T08:46:15Z TRACE twoliter::project::lock::image] get_manifest; self=ImageResolver { image: ProjectImage { image: Image { name: ValidIdentifier("bottlerocket-kernel-kit"), version: Version { major: 1, minor: 0, patch: 2 }, vendor: ValidIdentifier("bottlerocket") }, vendor: Verbatim(VerbatimVendor { vendor_name: ValidIdentifier("bottlerocket"), vendor: Vendor { registry: "public.ecr.aws/bottlerocket" } }) }, skip_metadata_retrieval: false } image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } } image=bottlerocket-kernel-kit-1.0.2@public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2 uri=public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2
[2025-01-24T08:46:15Z DEBUG twoliter::project::lock::image] Fetching image manifest. image=bottlerocket-kernel-kit-1.0.2@public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2 uri="public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2"
[2025-01-24T08:46:17Z TRACE twoliter::project::lock::image] calculate_digest; self=ImageResolver { image: ProjectImage { image: Image { name: ValidIdentifier("bottlerocket-kernel-kit"), version: Version { major: 1, minor: 0, patch: 2 }, vendor: ValidIdentifier("bottlerocket") }, vendor: Verbatim(VerbatimVendor { vendor_name: ValidIdentifier("bottlerocket"), vendor: Vendor { registry: "public.ecr.aws/bottlerocket" } }) }, skip_metadata_retrieval: false } image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } } image=bottlerocket-kernel-kit-1.0.2@public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2 uri=public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2
[2025-01-24T08:46:18Z DEBUG twoliter::project::lock::image] Calculated digest for locked image 'public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2': 'LpwDWPrnz82tSbZF87jkcppRuqIkJ48EObt1MaNmVqU='
[2025-01-24T08:46:18Z DEBUG twoliter::project::lock::image] Extracting kit metadata from OCI image
[2025-01-24T08:46:18Z TRACE twoliter::project::lock::image] try_from_image; image_uri="public.ecr.aws/bottlerocket/bottlerocket-kernel-kit@sha256:b5ff1be5bb656218ed9d4088d6af8d755cdf8b404d53a203886aa6bb1714501e" image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } }
[2025-01-24T08:46:18Z TRACE twoliter::project::lock::image] Extracting kit metadata from OCI image config image_uri="public.ecr.aws/bottlerocket/bottlerocket-kernel-kit@sha256:b5ff1be5bb656218ed9d4088d6af8d755cdf8b404d53a203886aa6bb1714501e"
[2025-01-24T08:46:20Z TRACE twoliter::project::lock::image] Kit metadata retrieved from image config image_uri="public.ecr.aws/bottlerocket/bottlerocket-kernel-kit@sha256:b5ff1be5bb656218ed9d4088d6af8d755cdf8b404d53a203886aa6bb1714501e" image_config=ConfigView { labels: {"dev.bottlerocket.kit.v2": "eyJraXQiOltdLCJuYW1lIjoiYm90dGxlcm9ja2V0LWtlcm5lbC1raXQiLCJzZGsiOnsiZGlnZXN0IjoiUmpweS9nVmdCaFUvQjY5NnhhSzFZNC9kcno0cE5KdStmeXlaU0lrOW9MRT0iLCJuYW1lIjoiYm90dGxlcm9ja2V0LXNkayIsInNvdXJjZSI6InB1YmxpYy5lY3IuYXdzL2JvdHRsZXJvY2tldC9ib3R0bGVyb2NrZXQtc2RrOnYwLjUwLjAiLCJ2ZW5kb3IiOiJib3R0bGVyb2NrZXQiLCJ2ZXJzaW9uIjoiMC41MC4wIn0sInZlcnNpb24iOiIxLjAuMiJ9Cg=="} } kit_metadata=<ImageMetadata(decoded) [ImageMetadata { name: "bottlerocket-kernel-kit", version: Version { major: 1, minor: 0, patch: 2 }, sdk: Image { name: ValidIdentifier("bottlerocket-sdk"), version: Version { major: 0, minor: 50, patch: 0 }, vendor: ValidIdentifier("bottlerocket") }, kits: [] }]>
[2025-01-24T08:46:20Z TRACE twoliter::project::lock::image] Checking that all manifests refer to the same kit.
[2025-01-24T08:46:20Z TRACE twoliter::project::lock::image] try_from_image; image_uri="public.ecr.aws/bottlerocket/bottlerocket-kernel-kit@sha256:45238ee2b985994948b68e1b1390720ea1a1f2ababbb67e2cdb74a15eeca82b4" image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } }
[2025-01-24T08:46:20Z TRACE twoliter::project::lock::image] Extracting kit metadata from OCI image config image_uri="public.ecr.aws/bottlerocket/bottlerocket-kernel-kit@sha256:45238ee2b985994948b68e1b1390720ea1a1f2ababbb67e2cdb74a15eeca82b4"
[2025-01-24T08:46:22Z TRACE twoliter::project::lock::image] Kit metadata retrieved from image config image_uri="public.ecr.aws/bottlerocket/bottlerocket-kernel-kit@sha256:45238ee2b985994948b68e1b1390720ea1a1f2ababbb67e2cdb74a15eeca82b4" image_config=ConfigView { labels: {"dev.bottlerocket.kit.v2": "eyJraXQiOltdLCJuYW1lIjoiYm90dGxlcm9ja2V0LWtlcm5lbC1raXQiLCJzZGsiOnsiZGlnZXN0IjoiUmpweS9nVmdCaFUvQjY5NnhhSzFZNC9kcno0cE5KdStmeXlaU0lrOW9MRT0iLCJuYW1lIjoiYm90dGxlcm9ja2V0LXNkayIsInNvdXJjZSI6InB1YmxpYy5lY3IuYXdzL2JvdHRsZXJvY2tldC9ib3R0bGVyb2NrZXQtc2RrOnYwLjUwLjAiLCJ2ZW5kb3IiOiJib3R0bGVyb2NrZXQiLCJ2ZXJzaW9uIjoiMC41MC4wIn0sInZlcnNpb24iOiIxLjAuMiJ9Cg=="} } kit_metadata=<ImageMetadata(decoded) [ImageMetadata { name: "bottlerocket-kernel-kit", version: Version { major: 1, minor: 0, patch: 2 }, sdk: Image { name: ValidIdentifier("bottlerocket-sdk"), version: Version { major: 0, minor: 50, patch: 0 }, vendor: ValidIdentifier("bottlerocket") }, kits: [] }]>
[2025-01-24T08:46:22Z DEBUG twoliter::project::lock] Resolving workspace SDK sdk_set={ProjectImage { image: Image { name: ValidIdentifier("bottlerocket-sdk"), version: Version { major: 0, minor: 50, patch: 0 }, vendor: ValidIdentifier("bottlerocket") }, vendor: Verbatim(VerbatimVendor { vendor_name: ValidIdentifier("bottlerocket"), vendor: Vendor { registry: "public.ecr.aws/bottlerocket" } }) }}
[2025-01-24T08:46:22Z DEBUG twoliter::project::lock] Resolving workspace SDK sdk=ProjectImage { image: Image { name: ValidIdentifier("bottlerocket-sdk"), version: Version { major: 0, minor: 50, patch: 0 }, vendor: ValidIdentifier("bottlerocket") }, vendor: Verbatim(VerbatimVendor { vendor_name: ValidIdentifier("bottlerocket"), vendor: Vendor { registry: "public.ecr.aws/bottlerocket" } }) }
[2025-01-24T08:46:22Z TRACE twoliter::project::lock::image] resolve; self=ImageResolver { image: ProjectImage { image: Image { name: ValidIdentifier("bottlerocket-sdk"), version: Version { major: 0, minor: 50, patch: 0 }, vendor: ValidIdentifier("bottlerocket") }, vendor: Verbatim(VerbatimVendor { vendor_name: ValidIdentifier("bottlerocket"), vendor: Vendor { registry: "public.ecr.aws/bottlerocket" } }) }, skip_metadata_retrieval: true } image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } } image=bottlerocket-sdk-0.50.0@public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0 uri=public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0
[2025-01-24T08:46:22Z INFO  twoliter::project::lock::image] Resolving dependency image dependency 'bottlerocket-sdk-0.50.0@public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0'.
[2025-01-24T08:46:22Z TRACE twoliter::project::lock::image] get_manifest; self=ImageResolver { image: ProjectImage { image: Image { name: ValidIdentifier("bottlerocket-sdk"), version: Version { major: 0, minor: 50, patch: 0 }, vendor: ValidIdentifier("bottlerocket") }, vendor: Verbatim(VerbatimVendor { vendor_name: ValidIdentifier("bottlerocket"), vendor: Vendor { registry: "public.ecr.aws/bottlerocket" } }) }, skip_metadata_retrieval: true } image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } } image=bottlerocket-sdk-0.50.0@public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0 uri=public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0
[2025-01-24T08:46:22Z DEBUG twoliter::project::lock::image] Fetching image manifest. image=bottlerocket-sdk-0.50.0@public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0 uri="public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0"
[2025-01-24T08:46:23Z TRACE twoliter::project::lock::image] calculate_digest; self=ImageResolver { image: ProjectImage { image: Image { name: ValidIdentifier("bottlerocket-sdk"), version: Version { major: 0, minor: 50, patch: 0 }, vendor: ValidIdentifier("bottlerocket") }, vendor: Verbatim(VerbatimVendor { vendor_name: ValidIdentifier("bottlerocket"), vendor: Vendor { registry: "public.ecr.aws/bottlerocket" } }) }, skip_metadata_retrieval: true } image_tool=ImageTool { image_tool_impl: CraneCLI { cli: CommandLine { path: "/tmp/.tmpRMXSfc/krane" } } } image=bottlerocket-sdk-0.50.0@public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0 uri=public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0
[2025-01-24T08:46:25Z DEBUG twoliter::project::lock::image] Calculated digest for locked image 'public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0': 'Rjpy/gVgBhU/B696xaK1Y4/drz4pNJu+fyyZSIk9oLE='
[2025-01-24T08:46:25Z DEBUG twoliter::project::lock] Comparing resolved lock to current lock state current_lock=Lock { schema_version: 1, sdk: LockedImage { name: ValidIdentifier("bottlerocket-sdk"), version: Version { major: 0, minor: 50, patch: 0 }, vendor: ValidIdentifier("bottlerocket"), source: "public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0", digest: "Rjpy/gVgBhU/B696xaK1Y4/drz4pNJu+fyyZSIk9oLE=" }, kit: [LockedImage { name: ValidIdentifier("bottlerocket-core-kit"), version: Version { major: 5, minor: 3, patch: 0 }, vendor: ValidIdentifier("bottlerocket"), source: "public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0", digest: "U6fyce19tV0QNvUux86If1eEQY8k/7M62sEsGZCI7Wc=" }, LockedImage { name: ValidIdentifier("bottlerocket-kernel-kit"), version: Version { major: 1, minor: 0, patch: 2 }, vendor: ValidIdentifier("bottlerocket"), source: "public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2", digest: "LpwDWPrnz82tSbZF87jkcppRuqIkJ48EObt1MaNmVqU=" }] } resolved_lock=Lock { schema_version: 1, sdk: LockedImage { name: ValidIdentifier("bottlerocket-sdk"), version: Version { major: 0, minor: 50, patch: 0 }, vendor: ValidIdentifier("bottlerocket"), source: "public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0", digest: "Rjpy/gVgBhU/B696xaK1Y4/drz4pNJu+fyyZSIk9oLE=" }, kit: [LockedImage { name: ValidIdentifier("bottlerocket-core-kit"), version: Version { major: 5, minor: 3, patch: 0 }, vendor: ValidIdentifier("bottlerocket"), source: "public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0", digest: "U6fyce19tV0QNvUux86If1eEQY8k/7M62sEsGZCI7Wc=" }, LockedImage { name: ValidIdentifier("bottlerocket-kernel-kit"), version: Version { major: 1, minor: 0, patch: 2 }, vendor: ValidIdentifier("bottlerocket"), source: "public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2", digest: "LpwDWPrnz82tSbZF87jkcppRuqIkJ48EObt1MaNmVqU=" }] }
[2025-01-24T08:46:25Z TRACE twoliter::project::lock::verification] write_tags; self=VerificationTagger { tags: {Sdk(VerificationManifest { verified_images: {"bottlerocket-sdk-0.50.0@bottlerocket (public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0)"} }), Kits(VerificationManifest { verified_images: {"bottlerocket-core-kit-5.3.0@bottlerocket (public.ecr.aws/bottlerocket/bottlerocket-core-kit:v5.3.0)", "bottlerocket-kernel-kit-1.0.2@bottlerocket (public.ecr.aws/bottlerocket/bottlerocket-kernel-kit:v1.0.2)"} })} }
[2025-01-24T08:46:25Z DEBUG twoliter::project::lock::verification] Cleaning up any existing tag files for resolved artifacts
[2025-01-24T08:46:25Z DEBUG twoliter::project::lock::verification] Writing tag files for verified artifacts
[2025-01-24T08:46:25Z DEBUG twoliter::project::lock::verification] Writing tag file for verified artifacts: '/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/external-kits/.sdk-verified'
[2025-01-24T08:46:25Z DEBUG twoliter::project::lock::verification] Writing tag file for verified artifacts: '/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/external-kits/.kits-verified'
[2025-01-24T08:46:25Z DEBUG twoliter::tools] Installing tools to '/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools'
[2025-01-24T08:46:25Z TRACE twoliter::common::fs] remove_dir_all; path=/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools
[2025-01-24T08:46:25Z TRACE twoliter::common::fs] create_dir_all; path=/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools
[2025-01-24T08:46:25Z DEBUG twoliter::tools] Installed tools to '/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools'
[2025-01-24T08:46:25Z TRACE twoliter::common::fs] metadata; path=/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools/build.Dockerfile
[2025-01-24T08:46:25Z TRACE twoliter::common::fs] copy; from=/tmp/.tmpRMXSfc/krane to=/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools/krane
[2025-01-24T08:46:25Z TRACE twoliter::common] exec; cmd=Command { std: "cargo" "make" "--disable-check-for-updates" "--makefile" "/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools/Makefile.toml" "--cwd" "/home/mikn/devel/molnett/src/platform/base/bottlerocket" "-e=TLPRIVATE_SDK_IMAGE=public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0" "-e=BUILDSYS_OUTPUT_GENERATION_ID=1" "-e=TWOLITER_TOOLS_DIR=/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools" "-e=BUILDSYS_ARCH=x86_64" "-e=BUILDSYS_KIT=platform-kit" "-e=BUILDSYS_VERSION_IMAGE=1.0.0" "-e=GO_MODULES=" "-e=BUILDSYS_UPSTREAM_SOURCE_FALLBACK=false" "build-kit", kill_on_drop: false } quiet=false
[2025-01-24T08:46:25Z DEBUG twoliter::common] Running: Command { std: "cargo" "make" "--disable-check-for-updates" "--makefile" "/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools/Makefile.toml" "--cwd" "/home/mikn/devel/molnett/src/platform/base/bottlerocket" "-e=TLPRIVATE_SDK_IMAGE=public.ecr.aws/bottlerocket/bottlerocket-sdk:v0.50.0" "-e=BUILDSYS_OUTPUT_GENERATION_ID=1" "-e=TWOLITER_TOOLS_DIR=/home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools" "-e=BUILDSYS_ARCH=x86_64" "-e=BUILDSYS_KIT=platform-kit" "-e=BUILDSYS_VERSION_IMAGE=1.0.0" "-e=GO_MODULES=" "-e=BUILDSYS_UPSTREAM_SOURCE_FALLBACK=false" "build-kit", kill_on_drop: false }
[cargo-make] INFO - cargo make 0.37.23
[cargo-make] INFO - 
[cargo-make] INFO - Build File: /home/mikn/devel/molnett/src/platform/base/bottlerocket/build/tools/Makefile.toml
[cargo-make] INFO - Task: build-kit
[cargo-make] INFO - Profile: development
[cargo-make] INFO - Running Task: check-cargo-version
[cargo-make] INFO - Running Task: setup
[cargo-make] INFO - Running Task: setup-build
[cargo-make] INFO - Running Task: fetch-sdk
[cargo-make] INFO - Running Task: fetch-sources
[cargo-make] INFO - Running Task: fetch-vendored
[cargo-make] INFO - Running Task: publish-setup
08:46:25 [INFO] No infra config at '/home/mikn/devel/molnett/src/platform/base/bottlerocket/Infra.toml' - using local roles/keys
[cargo-make] INFO - Running Task: cargo-metadata
[cargo-make] INFO - Running Task: validate-kits
[cargo-make] INFO - Running Task: build-kit
    Finished `dev` profile [optimized] target(s) in 0.01s
[cargo-make] INFO - Build Done in 0.53 seconds.
mikn@hal8999 ~/d/m/s/p/b/bottlerocket ((4677390c))>

Generally, it seems that krane is taking up the time when talking to your repository servers. This will probably be fixed on our end when we start hosting our own OCI artifacts for the kits and SDK, but it also spends surprisingly long on verifying the digests of the OCI artifacts, and for some reason it checks two kernel kits?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mikn @cbgbt