request: add populate method

[IAlphaOmegaI]

I have this case that I think is applicable to a lot of people:

Below is a table in my schema:

messages: defineTable({
    room: v.id("rooms"),
    replyOf: v.id("messages"),
    content: v.union(
      v.object({
        type: v.literal("text"),
        text: v.string(),
      }),
      v.object({
        type: v.literal("image"),
        thumbnailUrl: v.string(),
        url: v.string(),
      }),
      v.object({
        type: v.literal("video"),
        url: v.string(),
        thumbnailUrl: v.string(),
        duration: v.number(),
      }),
      v.object({
        type: v.literal("voice"),
        url: v.string(),
        duration: v.number(),
      }),
      v.object({
        type: v.literal("location"),
        location: v.object({
          latitude: v.float64(),
          longitude: v.float64(),
        }),
      }),
    ),
  }),

and here is a function that paginates messages

export const getRoomMessages = query({
  args: { pager: paginationOptsValidator, roomId: v.id("rooms") },
  handler: async (ctx, { pager, roomId }) => {
    const rooms = await ctx.runQuery(api.services.chat.getUserRooms);
    const isUserPartOfRoom =
      rooms.find((room) => room._id === roomId) !== undefined;
    if (!isUserPartOfRoom)
      throw new ConvexError("User is not in requested room");

    return await ctx.db
      .query("messages")
      .filter((q) => q.eq(q.field("room"), roomId))
      .paginate(pager);
  },
});

it would be really cool if in the paginate or query methods there existed a populate method that i could use to populate the replyOf id field, similar to mongoose and mongo db

[thomasballinger]

Populate sounds like convenient sugar for doing the DB lookup and replacing the foreign key in the object with the object the foreign key refers to, so doing this in fewer lines of code? Typically this is done today by building objects with JavaScript, but some syntax explorations in https://labs.convex.dev/convex-ents show something like populate.

Is this right, you're looking for a shorter way to write this code? I'm confused because this code doesn't seem to be populating, it's just returning the relevant messages for the room ID. Also note that you probably want to use indexes here for efficiency.

You might find these articles useful

• https://stack.convex.dev/relationship-structures-let-s-talk-about-schemas
• https://stack.convex.dev/functional-relationships-helpers

[IAlphaOmegaI]

Populate sounds like convenient sugar for doing the DB lookup and replacing the foreign key in the object with the object the foreign key refers to, so doing this in fewer lines of code? Typically this is done today by building objects with JavaScript, but some syntax explorations in https://labs.convex.dev/convex-ents show something like populate.

Is this right, you're looking for a shorter way to write this code? I'm confused because this code doesn't seem to be populating, it's just returning the relevant messages for the room ID. Also note that you probably want to use indexes here for efficiency.

You might find these articles useful

• https://stack.convex.dev/relationship-structures-let-s-talk-about-schemas
• https://stack.convex.dev/functional-relationships-helpers

Yes your right, I should've shown this code which actually would benefit from populate

 const messages: Message[] = await Promise.all(
      result.page.map(async (message, i) => {
        const previousMessage = result.page[i - 1];
        const isLastMessageInChain =
          !previousMessage || previousMessage.sender !== message.sender;
        const { content: raw } = message;

        const content = await (async () => {
          switch (raw.type) {
            case "video":
            case "image":
              return {
                ...raw,
                url: await ctx.storage.getUrl(raw.id),
                thumbnailUrl: await ctx.storage.getUrl(raw.thumbnailId),
              };
            case "audio":
              return {
                ...raw,
                url: await ctx.storage.getUrl(raw.id),
              };
            case "text":
            case "location":
              return raw;
          }
        })();

        if (!message.replyOf) {
          return { ...message, isLastMessageInChain } as Message;
        }

        const replyOf = await ctx.db.get(message.replyOf);
        if (!replyOf) return { ...message, isLastMessageInChain } as Message;

        return {
          ...message,
          replyOf,
          content,
          isLastMessageInChain,
        } as Message;
      }),
    );
    return {
      ...result,
      page: messages,
    };
  },

specifically the replyOf section

  const replyOf = await ctx.db.get(message.replyOf);
      if (!replyOf) return { ...message, isLastMessageInChain } as Message;

      return {
        ...message,
        replyOf,
        content,
        isLastMessageInChain,
      }  as Message;
      

this could've been much easier if .populate("replyOf") existed, exactly as you described.