Adding Heroku's buildpacks as a builder option #795
Replies: 3 comments 1 reply
-
|
Thanks for posting the discussion @nickhammond. I wrote the linked article and I'm the buildpack maintainer. I'm happy to answer any CNB or Ruby buildpack-related questions the Kamal core team might have.
There's a The Also if anyone plays around with buildpacks and has usability feedback, please post on our discussion. @nickhammond's first hand experience has been hugely valuable over at heroku/buildpacks#6. |
Beta Was this translation helpful? Give feedback.
-
|
@schneems I started testing out deploying with Kamal via buildpacks by utilizing the This almost works but Kamal relies heavily on labels to figure out which containers belong to which service so that it knows it's telling the correct container to stop, start, etc. Kamal currently does this with the Is it possible to set labels within project.toml? I found this placeholder doc page https://buildpacks.io/docs/for-buildpack-authors/how-to/write-buildpacks/add-labels/ and I'm not seeing anything obvious in the project.toml spec. I tried this out with Paketo and it looks like they have an image-labels buildpack specifically for this. Is there something similar for Heroku's buildpacks? |
Beta Was this translation helpful? Give feedback.
-
|
Before adding an additional builder to Kamal I wanted to do a simple prototype utilizing the Note that a Dockerfile is not necessary, the build-specific artifacts would be the project.toml. This is where you set the buildpacks that your app needs(ruby, node, python, etc), the required label for Kamal, and the files that can be excluded from the final image. Here's a rough overview to get an app deployed via buildpacks with Kamal, I'll probably write a longer blog post about it at some point but just wanted to share the core pieces of it. pre-connect - This would be what the new builder ends up needing to do. config/deploy.yml - buildpacks don't expose a port, you have to publish the correct port for your container. Procfile - The default process type is project.toml I also set my default builder to heroku-24( With this just building on Utilizing buildpacks then gives you the familiar Heroku auto-detection, cache, and build process that you're probably familiar with. You would trigger this flow by using the skip push option( pack build --platform linux/amd64 nickhammond/hey -t nickhammond/hey:fbbfc237711ba384fee3c4d40e5b758fae51d292 |
Beta Was this translation helpful? Give feedback.
-
|
@schneems PR opened here #916. Could use your insight on using secrets for the build process and ensuring those don't end up in the final image. It'd also be great to be able to point to a .dockerignore file for the Exclude list in project.toml, haven't done anything around that yet. |
Beta Was this translation helpful? Give feedback.
-
Heroku recently open-sourced their Cloud Native Buildpack and it's worth exploring as an alternative builder option. This project is essentially the Heroku build process, a very familiar build process that just figures out a bunch of the basic stuff for you by checking for various files within your app.
Docker often comes up as a stumbling block on Discord and GitHub discussions for Kamal as people transition from other PAAS offerings or perhaps provisioned VPS machines. There's also a lot of reinventing the wheel for things like caching asset compilation or bundle install caching with Docker builds.
The Heroku buildpacks have been doing this for a long time and it's also a shared effort to get a Ruby, Node, Python, etc. app hosted. There are 9 officially supported buildpacks from Heroku but another 7800 from the community that you can leverage with Heroku buildpacks. The buildpacks are also Cloud Native Buildpacks which is now an open spec by the Cloud Native Computing Foundation (CNCF) and no longer bespoke to Heroku.
Currently with Kamal we can almost accomplish this with a
pre-buildscript that packs our app and then references the packed image in ourdeploy.ymlthat we just pushed up to our registry. Something like this:We'd want to also tag the container with the
KAMAL_VERSIONwith these steps.Packing our app results in a fully built OCI-compliant container image that we can then push up to our registry host. For example, here's a build for a Rails app with the
heroku-22buildpack:First build output
Second build output
The first pack takes 1:35 while the second pack takes only 14 seconds total. Building this same image fresh with Docker takes around 2:50 which is in the normal build time range for building a Rails app with Docker. A second build for this particular app with just application-level changes comes to about 35 seconds. The Dockerfile for this app has also been optimized as much as possible with a multi-stage build which also includes caching installed gems and apt packages. Sure, there's always improvements you can make with more caching and more optimized build steps with Docker but the buildpacks give us all of these optimizations out of the box. The bigger win out of this though is that we didn't have to deal with optimizing the build, we were just able to pack it which utilizes all of the buildpack detection support.
Right now with Kamal you have to build an image from your app either locally or with a remote builder with Docker. The idea would be to introduce an approach that builds your app with buildpacks on the backend instead of Docker which also means you don't need a
Dockerfile.Maybe something like this:
Or:
Pack seems a little more obvious what it's doing but it also might make sense to have a setting for
builderthat can be set todockerorcnb(?). Additional customizations to the pack process can be set in a project.toml file. There's probably a few additional pack-related configuration settings to bubble up but it'd be great to keep as much of that within the pack config, similar to other package managers like bundler, yarn, etc.Currently the built image with
heroku-22is a bit bloated butheroku-24is being actively worked on and is showing significant improvements to the final image size.It's worth noting that this wouldn't replace the default Docker build process but an alternative option that can be simpler to get your app built with. If you want to ensure an incredibly lean and fully customized image you can always reach for Docker but if you're just getting started with Kamal or it's a standardized app you can basically follow the path that Heroku has been busy paving for us. Kamal is built around being able to run a container with Docker but that container doesn't necessarily have to be built with Docker and a Cloud Native Buildpack seems like a nice alternative to introduce.
Resources:
Beta Was this translation helpful? Give feedback.
All reactions