Build an AI Image Generator With Rowy (2023)

Humans are visual animals, so a tool like Stable Diffusion that can generate images from text prompts is a game changer for any online business relying on visual content.

But using a generative AI tool comes with technical challenges that can seem daunting at first. In this tutorial, we'll show you how to use Rowy to build a simple app that's as simple as using a spreadsheet, with the ability to generate images from text prompts, and then upscale the result image.

Note that you can use the full Image Generation demo in our demo page as well.

1. Create A New Rowy Table

You'll need a Rowy account to create a smart spreadsheet. The installation process only takes a few minutes and guides you step-by-step: install Rowy.

You can also self-host Rowy using the Github open-source version.

Once you're logged in, create a new project and a new table. You can name it "Image Generation" for example and add a description to explain what it's for.

2. Write A Prompt

We need to add some basic columns to store the data we need. Stable Diffusion takes a text prompt to generate an image, so we'll have a promptText column. You can also tell Stable Diffusion to use an image prompt to build upon it, and an artistic style to apply to the result image. That gives us three columns in total:

The promptImage column has an Image type, and the pickAStyle column can be a Multi-Select field with pre-defined values like cyberpunk, photorealistic, neon lights for example.

3. Generate An Image With Rowy Actions

Then we generate the actual images using an Action column called click to generate. An Action column is a special column type that can run a custom script when you click on its button. First, create a new column with the Action type:

An action script comes with pre-defined variables like a row object to access the current row's data or the ref object to access the table's data. We use ReplicateAI to abstract Stable Diffusion in a convenient API request and process the result:

const action: Action = async ({ row, ref, db, storage, auth, actionParams, user }) => {

    const replicateKey = await rowy.secrets.get('replicate')

    const { style } = row

    let prompt = row.prompt

    if (style){
        prompt += ' ' + style
    }

    const requestBody = { 
        "version": "27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478", 
        "input": { 
            prompt,
            num_outputs: actionParams.multiple? 4:1
        },
        webhook_completed:"https://rowy-hooks-wylpw2t4lq-uc.a.run.app/wh/imageGeneration/DHblWpTqCBDoNfDKiipz",
    }

    if (row.initImage && row.initImage[0] && row.initImage[0].downloadURL){
        requestBody.input.init_image = row.initImage[0].downloadURL
        requestBody.input.prompt_strength = 0.75
    }

    const res = await fetch("https://api.replicate.com/v1/predictions", {
        body: JSON.stringify(requestBody),
        headers: {
        Authorization: `Token ${replicateKey}`,
        "Content-Type": "application/json"
        },
        method: "POST"
    })

    if (!res.ok){
        return {
            message: `failed request ${res.status} ${res.statusText}`,
            success: false
        }
    }

    const result = await res.json()

    await ref.update({ stableDiffusion : result })

    return {
        message: `Model Started generating ${prompt}`,
        success: true
    }
}

All you have to do is paste the aforementionned code in the column settings, under the Action section:

Now, every time you click on the button, the script will run.

4. Getting The Result Image Via Webhook

Stable Diffusion is a long-running process, so we need to use a webhook to get the result image. A webhook is simply an HTTP endpoint that you can call to trigger a script. In our case, the webhook will be called by ReplicateAI when the model is done generating the image. You'll need a new column stableDiffusion to store the responses from ReplicateAI.

Rowy makes creating webhooks as easy as clicking on the Webhooks icon, like so:

Just like an Action column, you can then paste custom script code in the webhook's settings:

const basicParser: Parser = async({ req, db, ref }) => {
    const job_id = req.body.id

    const query = await ref.where("stableDiffusion.id", "==", job_id).get()

    if (query.empty){
        return null
    } 

    const _updatedBy = await rowy.metadata.serviceAccountUser()

    await query.docs[0].ref.update({
        stableDiffusion:body, 
        _updatedBy
    })

    return null 
}

You'll obtain something like this:

Stable Diffusion will send HTTP events to your webhook whenever it's making any progress. You just need to wait for a completed event to get the result image.

We can then create a new column to display the result image. To do that we use a Derivative column that will monitor changes in the stableDiffusion column and extract the link from ReplicateAI's HTTP responses. A Derivative column is a special column type that can be used to extract data from other columns. You can create a new Derivative column and paste the following code:

const derivative: Derivative = async ({ row, ref, db, storage, auth }) => {
    const imageUrls = row.stableDiffusion?.output

    if(!imageUrls){
        return []
    }

    return Promise.all(imageUrls.map((url, i) => {
        return rowy.storage.upload.url(url, {
            folderPath: ref.path + `/stableDiffusion/`,
            fileName: row.stableDiffusion.id + '-' + i + '.png'
        })
    }))
}

We built a working prototype! You can now click on the click to generate button to generate an image:

5.Upscale The Result Image

When you find a result image you like, you can ask Stable Diffusion to upscale it to a higher resolution. Once again, you just need an Action column to call the API and a webhook to get the result image:

const action: Action = async ({ row, ref, db, storage, auth, actionParams, user }) => {
  const replicateKey = await rowy.secrets.get('replicate')

  const res = await fetch("https://api.replicate.com/v1/predictions", {
    body: JSON.stringify({ "version": "42fed1c4974146d4d2414e2be2c5277c7fcf05fcc3a73abf41610695738c1d7b", "input": { "image": row.sdSamples[actionParams.image-1].downloadURL},
      webhook_completed:"https://rowy-hooks-wylpw2t4lq-uc.a.run.app/wh/imageGeneration/yqCcueUbNEofAQd7BUQX",
      scale:actionParams.scale
     }
    ),
    headers: {
      Authorization: `Token ${replicateKey}`,
      "Content-Type": "application/json"
    },
    method: "POST"
  })

  const result = await res.json()

  await ref.update({ upscaleState: result })

  return {
    message: "Model Started",
    success: true
  }
}

And the webhook:

const basicParser: Parser = async({ req, db, ref }) => {
    const {body} = req;

    const _updatedBy = await rowy.metadata.serviceAccountUser()
    const {id} = body

    const query = await ref.where("upscaleState.id","==",id).get()

    if (query.empty){
        return null
    } else {
        await query.docs[0].ref.update({upscaleState:body,_updatedBy})

        return null 
    }
}

Which gives us the final result:

Try More Demos

Hope you enjoyed it! If you'd like to see more examples of what you can do with Rowy, you can visit this link: https://demo.rowy.io/tables. There are 24 demos and examples available, including using Replicate API for face restoration and GPT-3 for text-to-speech. If you have any questions about building something with Rowy, don't hesitate to reach out to us. We are happy to assist you!

Share this post