Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sessions #1055

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
310 changes: 310 additions & 0 deletions proposals/0054-sessions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,310 @@
<!--
Note: You are probably looking for `stage-1--discussion-template.md`!
This template is reserved for anyone championing an already-approved proposal.
Community members who would like to propose an idea or feature should begin
by creating a GitHub Discussion. See the repo README.md for more info.
To use this template: create a new, empty file in the repo under `proposals/${ID}.md`.
Replace `${ID}` with the official accepted proposal ID, found in the GitHub Issue
of the accepted proposal.
-->

**If you have feedback and the feature is released as experimental, please leave
it on the Stage 3 PR. Otherwise, comment on the Stage 2 issue (links below).**

- Start Date: 2024-11-15
- Reference Issues: <!-- related issues, otherwise leave empty -->
- Implementation PR: https://github.com/withastro/astro/pull/12441
- Stage 2 Issue: https://github.com/withastro/roadmap/issues/1050
- Stage 3 PR: https://github.com/withastro/roadmap/pull/1055
<!-- related roadmap PR, leave it empty if you don't have a PR yet -->

# Summary

A first class `Astro.session` primitive, with pluggable storage backends.
Inspired by PHP sessions and Rails sessions, this will allow on-demand rendered
pages and API endpoints to use a new `Astro.session` object to access arbitrary
data, scoped to that browser session. A session ID cookie is used to associate
the browser with the session, but the actual data is stored on the backend.

# Example

Usage in an Astro component:

```astro
---
// src/components/CartButton.astro
export const prerender = false; // Not needed in 'server' mode
const cart = await Astro.session.get('cart');
---
<a href="/checkout">🛒 {cart?.length ?? 0} items</a>
```

Usage in an API endpoint:

```ts
// src/pages/api/addToCart.ts
import type { APIContext } from "astro";

export async function POST(req: Request, context: APIContext) {
const cart = await context.session.get("cart");
cart.push(req.body.item);
await Astro.session.set("cart", cart);
return Response.json(cart);
}
```

Usage in an Action:

```ts
// src/actions/addToCart.ts
import { defineAction } from "astro:actions";
import { z } from "astro:schema";

export const server = {
addToCart: defineAction({
input: z.object({ productId: z.string() }),
handler: async (input, context) => {
const cart = await context.session.get("cart");
cart.push(input.productId);
await context.session.set("cart", cart);
return cart;
},
}),
};
```

# Background & Motivation

HTTP is a stateless protocol, so sharing data between requests is a problem that
most server-rendered web apps need to solve. The standard way is using cookies,
but these have limitations. Firstly, they are limited to 4kB in size, so can't
be used for anything complex. The full data also needs to be sent along with
every request and response, increasing their size. Finally, even when encrypted
they can be vulnerable to replay attacks. We have encountered the limitations of
cookie storage when building Astro Actions, where we need response data to
persist between page redirects. It is easy to reach the 4kB limit in these
cases.

For this reasons, most non-trivial apps need to implement some kind of
server-side session handling. In most cases these work by storing a random
session id in a cookie, and using that to retrieve the full session data on the
server. Monolithic frameworks such as Rails and Laravel normally implement
sessions at the framework or server level, and PHP has built-in session handling
which heavily influenced this proposal. However it isn't a common primitive for
modern JS frameworks, in part due to the fact that these need to work in
serverless environments, so simple default backends such as filesystem storage
cannot be relied upon. We propose addressing this by allowing adapters to
provide default implementations, relying on the primitives that they have
available.

# Goals

- Simple key/value API, similar to Astro cookies (`Astro.session.get('key')`,
`Astro.session.set('key', 'value')`)
- Accessible in Astro components, actions, and API endpoints via Astro global
and context objects.
- Additional control over the session via methods such as
`Astro.session.regenerate()` and `Astro.session.destroy()`
- Storage drivers for popular providers available out of the box.
- Adapters can specify a default storage driver, with Node using filesystem
storage by default.
- Other settings such as cookie name, session expiry etc are also optionally
configurable.
- Sessions are lazy-loaded and auto-generated when first accessed. Session ID
cookie is get and set automatically.

# Non-Goals

- Type-safety. This may be added later, but there are questions about the best
way to do this, because of issues with the way that the config is loaded.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course can be added later but adding it from the get-go would be nicer IMO.
To propose one option: Optional override in env.d.ts.

Ideally everything could be inferred from the Astro config in some way but in the mean time I think a sufficient alternative would be to allow users to add types to their src/env.d.ts to add explicit types for sessions. Similar to how it is done for Cloudflare types. For example, based on the examples above:

/// <reference path="../.astro/types.d.ts" />
/// <reference types="astro/client" />

declare namespace App {
	interface Sessions {
		cart: Array<string>;
	}
}

Or if long-term things were to get more advanced some kind of schema could be added as an option to the session config, similar to server actions, to both validate data & infer types from.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You must distinguish between typed and type-safe, sessions must be assumed unreliable and partial in the best of circumstances, enforcing it with a schema is just a trap for when the sessions start to skew or expire and suddenly no longer fit. I think either a FormData-like API (as suggested) or a typing that gets Partial-ized on the object is best, though it'll suck a bit ngl

- User management, auth etc. This is just raw sessions, but they may be useful
if someone is implementing auth
- Building or maintaining our own driver backends. We will rely on existing
libraries for this.
- Strong consistency, atomic writes

# Detailed Design

The session object is a wrapper around pluggable storage backends. It is
lazy-loaded: the contents of the session is fetched from the data store when
`get()` is first called, and is `set()` at the end of the request. This means
each request is treated as a single transaction, with read-your-writes
consistency through the local object. Persistent storage is last-write-wins,
with no attempt made to reconcile transactions between multiple requests. Each
backend has its own consistency model for the persistent storage, and where this
is configurable this will be exposed as part of the driver options.

## API

The session object is available in all Astro contexts, including components,
actions, and API endpoints. In components, it is accessed via the global `Astro`
object, and in actions and API endpoints it is available on the `context`
object. The API is the same in all cases.

Values are serialized and deserialized using
[devalue](https://github.com/Rich-Harris/devalue), which is the same library
used by content layer and actions. This means that supported types are the same,
and include strings, numbers, Dates, Maps, Sets, Arrays and plain objects.

### `Astro.session.get(key: string): Promise<any>`

Returns the value of the given key in the session. If the key does not exist, it
returns `undefined`.

### `Astro.session.set(key: string, value: any): void`

Sets the value of the given key in the session. The value can be any
serializable type.

### `Astro.session.regenerate(): void`

Regenerates the session ID. Best practice is to call this when a user logs in or
escalates their privileges, to prevent session fixation attacks.

### `Astro.session.destroy(): void`

Destroys the session, deleting the cookie and the object from the backaned. This
should be called when a user logs out or their session is otherwise invalidated.

## Configuration

The session object is configured using the `session` key in the Astro config.
All configuration is optional if the adapter has built-in support. This is the
case for the Node adapter, which uses filesystem storage by default, and is
expected to be the case for most adapters.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this should be part of the proposal, specifically the part when we talk about specific adapters. Or, if we really need to, we should try to actually to be as generic as possible because we don't know what and how the specific adapters will behave.


```js
// astro.config.ts
import { defineConfig } from "astro/config";
import { z } from "astro/schema";

export default defineConfig({
// Sessions require an adapter to be used
adapter: node({
mode: "standalone",
}),
session: {
// Required: the name of the Unstorage driver
driver: "redis",
Comment on lines +138 to +139
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have mixed feelings about using driver in a "shared" configuration. What if the redis driver isn't supported by a specific adapter or any other adapter? How can we tell the user that they can't use a certain adapter with certain driver?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by shared configuration?

Copy link
Member

@ematipico ematipico Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With "shared", I meant that If we define a driver here, I would assume - as a user - that this driver will work regardless of any adapter used by. So it's a configuration "shared" by core and the adapter at play

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's not such a tight relationship between an adapter and a driver. An adapter doesn't "support" a driver directly. It's more correct to say that some adapters do or do not support:

  • Persistent memory
  • FS access
  • TCP sockets

So for your example, it's not that some adapters support redis, it's that redis works over TCP sockets and some adapters may or may not support opening sockets. But we don't have such a feature map to know those sorts of requirements.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For any drivers that we aren't automatically enabling (so, basically all of them except fs) I think we need to delegate all the documentation to Unstorage, at least in the short term, and that includes which platforms are support. Anything else would be too hard to keep up to date. In future we may decide we want to own the drivers ourselves, but right now I think the idea is to rely on Unstorage wherever possible. If we need changes then we can contribute them upstream.

// The required options depend on the driver
options: {
base: "session",
host: process.env.REDIS_HOST,
password: process.env.REDIS_PASSWORD,
},
Comment on lines +140 to +145
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there are drivers that don't require options, don't options become optional? How do we validate options against the driver? Is it possible to do this at the schema level?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, options are optional. We don't have schemas for the options, but we have types, and those do correctly handle the different driver options.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I read // The required options and thought that they were required. So they are optional, but needs to be provided based on the driver.
Will we throw an error if those options are missing for specific drivers?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// The required options depend on the driver means some options will or will not be required, depending on the driver chosen.

The driver will throw the error if required options are missing.

// Optional: the name of the session ID cookie
cookieName: "my-session-id",
// Optional: cookie options
cookieOptions: {
// Default is 1 hour
maxAge: 86400,
// Default is 'Lax'
sameSite: "Strict",
},
ascorbic marked this conversation as resolved.
Show resolved Hide resolved
},
});
```

## Cookies

Sessions are generated when first accessed, and a session ID cookie is set in
the response. It can be regenerated at any time with
`Astro.session.regenerate()`, and destroyed with `Astro.session.destroy()`. The
ID is a 36-character v4 UUID generated using
[`crypto.randomUUID()`](https://developer.mozilla.org/en-US/docs/Web/API/Crypto/randomUUID)
from the Web Crypto API. The session cookie options are configurable, with the
defaults being `astro-session-id` for the name, and `HttpOnly` and
`SameSite=Lax` for the flags. The expiry is set to 1 hour by default, but this
is also configurable.

## Storage

The backend drivers are implemented using
[Unstorage](https://unstorage.unjs.io/), which provides a storage abstraction
with drivers for multiple providers.

The session is implemented as a single object stored in the backend, rather than
individual entries, to avoid the need to make multiple requests to the backend
for each read or write. This is a tradeoff between performance and consistency,
but is a common pattern in session handling. The session is fetched in full or
created when first accessed, and is written back in full at the end of the
request.

When `session.get()` is called, the session ID is read from the cookie. If there
is no session ID, a new one is generated. The full session data is then fetched
as a single value from the backend and deserialized into using devalue. Future
reads and writes within that request context are done on this local object.

When `session.set()` is called, if there is no session ID, a new one is
generated. If a local session object has already been fetched, the value is set
on that. If not, the data is set on a sparse local session object. This is
merged with the full session if `get()` is called later in the request.
Otherwise this is done at the end of the request, and the merged object is
serialized using devalue and written back to the backend.

## Adapter and integration support

Because of the variety of environments to which Astro can be deployed, there is
no single approach to storage that can be relied upon in all cases. For this
reason, adapters should provide default session storage drivers where possible.
Sessions are only available in on-demand rendered pages and API endpoints, so
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and API endpoints

This is misleading, because endpoints that look like src/pages/schema.json.js are run during the build, they create a physical file called src/pages/schea.json and they aren't run in SSR anymore.

The endpoint needs to be on-demand, too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "on-demand rendered" is meant to apply to "pages and API endpoints". I'll reword it to be less ambiguous.

there will always be an adapter available. The Node adapter will use filesystem
storage by default, but this is not suitable for serverless environments. For
these, the adapter can default to any storage service that is available. For
example, Netlify may use Netlify Blobs, Vercel may use Vercel KV or Upstash,
Cloudflare may use Cloudflare Workers KV and Deno Deploy may use Deno KV.
Integrations can also provide their own storage drivers, and these can be
auto-configured by the integration.
ematipico marked this conversation as resolved.
Show resolved Hide resolved

This is done using the normal integration API, and should be handled in the
[`astro:config:done` hook](https://docs.astro.build/en/reference/integrations-reference/#astroconfigdone).
The adapter or integration is responsible for ensuring that they do not
overwrite any user-defined driver configuration. Adapters may choose to accept
their own configuration options which they can apply to the storage driver where
needed. Adapters may provide a storage driver for use in development, or rely on
the built-in node adapter which is provided by the dev server.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should really highlight what would happen if drivers go in conflict, and if so how users/developers can do that. Is Astro responsible for that? Is the adapter for that? Is the user responsible for that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There can only be one driver, so I don't think there's an issue with conflicts there. Adapters are meant to prefer the user-defined drivers, though I don't think this is something we can enforce when adapters can do what they want to the config.

Copy link
Member

@ematipico ematipico Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a concrete example that comes to mind

A project with a Cloudflare adapter that attempts to use a Netlify blob. What would happen in this occasion? How does Astro behave? It isn't clear to me from the RFC:

import cloudflare from "@astrojs/cloudflare"
import {defineConfg} from "astro/config"

defineConfig({
	driver: 'netlify-blobs', // should use nstorage/drivers/netlify-blobs
	adapter: cloudflare()
})

Is it possible to have such a combination? If so, how? It doesn't seem likely

In another case, the use of an adapter that doesn't allow FS storage:

import cloudflare from "@astrojs/cloudflare" // let's assume `node:fs` isn't compatible with cloudflare
import {defineConfg} from "astro/config"

defineConfig({
	driver: 'fs',
	adapter: cloudflare()
})

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to do checks for people doing things that are clearly not allowed on that platform. Why would somebody choose Netlify blobs if they're not using Netlify? This isn't something we're doing automatically and hiding from them: this is equivalent to somebody choosing the Netlify adapter when they're deploying to Cloudflare, or using fs in a page on a site deployed to Cloudflare.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would somebody choose Netlify blobs if they're not using Netlify?

Mistakes, junior people who don't know very well what they're learning, possible differences between local behaviour and production behaviour, etc.

I would like to improve that DX if possible. Why can't an adapter accept a predefined list of supported storage?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, true it should work if you set the credentials manually

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthewp

I don't know if its true that you can't use the netlify-blobs driver with Cloudflare. If netlify-blobs operates over HTTP then more than likely it would work.

I don't know either, it was just an example to explain what I meant.

It would make more sense if they provided a list of drivers not supported

That's a step forward!


@ascorbic

I'd really like to avoid coupling us so closely to the Unstorage driver implementation. One of the design goals for this is to avoid needing to maintain them. If we did that, we'd need to update all adapters whenever they add a new driver, and theoretically audit them all manually whenever that happens.

I understand that, however we need to strive towards a decent enough DX and UX for our users. Here are some considerations:

  • We are responsible for the libraries we use as a service (see sharp, drizzle, etc.), and we always try to create a certain layer of compatibility for our users.
  • Since we don't state which library we rely on (rightfully), Astro is free to change the underneath library (if it ever happens) without changing the user facing APIs. For images, for example, we remove squoosh, but the APIs are unchanged - AFAIK.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a bigger question then. If we want to fully abstract away unstorage to the extent that we don't need to mention it then we need to document every driver ourselves. Now that may be a sensible thing to do, but it will mean we need to decide on a more limited list of ones that we will support.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We talked about this on a call, but I'll restate that we can't ensure that a user only uses compatible libraries with Astro. There are a number of places where a user might use a library that doesn't work:

  • Remark plugins
  • Vite plugins
  • Framework components

There's a tradeoff between protecting the user from mistakes and giving them the power to do what they want. I think this is a case where there's a large amount of compatibility already, and users will naturally use drivers that make sense for their host anyways. Adding restrictions would make the API less powerful and unnecessarily prevent legitimate use-cases.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unstorages raison d’être is to be the uncoupled interface and provide many implementations. It's used successfully by nitro and I can attest for its ergonomics. I don't think there's a compelling reason to reimplement it just to own the namespace it comes from

Regarding saving devs from themselves, obviously it comes from a good place but it's just a wild goose chase. The higher the guard rails, the more convoluted the hacks people would put in place to skip them. If a junior person who doesn't know very well what they're learning make a mistake then they would learn "the hard way". I personally believe that providing good descriptive errors is much more educational and effective than guardrails. Generally this is something astro is very good at, so it's not an odd expectation.

The suggested sessions API is already 99% existing. Add unstorage and use Astro.locals, mix in some cookie action and you're golden. The hard part of sessions was the async local context. With an RFC this obvious and implementation this lean it'll be a shame to bikeshed the issue when everyone involved can clearly see the value.

I hope it's ok that I butted in :)


# Testing Strategy

The session object API can be tested using a mock driver. e2e tests can be used
to test the dev server and cookie handling. Individual adapters can include
tests for their defaukt drivers.

# Drawbacks

- There is no universal fallback available for persistent storage, so adapters
must provide their own default drivers. Vercel KV is no longer automatically
available, so it does not have a baseline storage driver.
- Sessions can increase request latency, particularly if a slow storage backend
is used.
- Allowing arbitrary drivers may be a footgun, as users may choose drivers that
are unsuitable for session storage.

There are tradeoffs to choosing any path. Attempt to identify them here.

# Alternatives

- Allow users to implement their own session handling using Astro.cookie and
their own backends, such as unstorage.

# Adoption strategy

The initial release will be behind an experimental flag, and will be opt-in. It
will be a non-breaking change when introduced, and will not execute any code
unless the session object is accessed. We will implement automatic driver
support in the node adapter using the filesystem driver, with other adapters
added later.

# Unresolved Questions

How should edge middleware be handled?

# References

- [PHP Sessions](https://www.php.net/manual/en/intro.session.php)
- [Laravel sessions](https://laravel.com/docs/11.x/session)
- [How Rails sessions work](https://www.justinweiss.com/articles/how-rails-sessions-work/)
- [Securing Rails sessions](https://guides.rubyonrails.org/security.html#sessions)