Skip to main content

Tutorial: Building an AI Agent for Linear (Part 1)

· 29 min read

This tutorial will teach you how to build an AI agent that integrates with Linear to automatically find relevant resources (e.g., articles, tutorials, and documentation) whenever you create a new issue (Linear uses the term "issues" but you may be more familiar with the Agile term, "stories" – same thing).

Introduction​

But what does it actually mean to use an agent to find relevant resources for an issue? What will interaction with this agent look like? How will it share these resources?

Well, they say a picture is worth a thousand words...so how about a video?

Whenever you create a new issue in Linear, this AI agent will process the issue, search the web for relevant resources, and then post those resources as a comment on the issue. It's kind of like a supercharged version of Let Me Google That For You.

By the end of this tutorial, you'll have your own version that you can tinker with to your heart's content, as well as the knowledge and confidence to create other agents of your own.

Housekeeping​

Before we get started, let's get some things out of the way.

First, this tutorial doesn't assume any prior experience with creating agents, but you should be comfortable with TypeScript and Node.js. Familiarity with AWS wouldn't hurt either but it's not absolutely required; I'll do my best to explain the basics when we get there.

Second, to complete this tutorial, there are a few things you're going to need besides Node.js:

  • A Linear account
  • An Anthropic API Console account
  • An Anthropic API key
  • An AWS account
  • The AWS CLI and AWS CDK CLI

Don't worry if you don't have all of these right now. I'll let you know when you need them and link you to resources that will help you get set up.

Third, this tutorial is organized into three parts:

  1. Creating the agent (You are here) – Part 1 is entirely focused on using the Anthropic SDK to create an agent. You'll be able to run the agent on your machine without any dependencies on Linear or AWS. You're welcome to stop here if you just want to learn the basics of building your own agent. If that's you, I highly recommend also checking out Thorsten Ball's excellent post titled How to Build an Agent, without which this tutorial wouldn't have been possible.
  2. Deploying the agent (Coming Soon) – Part 2 will focus on, well, deploying. We'll deploy the agent to AWS using a serverless architecture, meaning that you won't be billed when you aren't using it. In addition, all of the resources will be deployed using infrastructure as code, so it'll be really easy to teardown and redeploy at will.
  3. Connecting with Linear (Coming Soon) – Part 3 will cover how to link the agent with your Linear account so that it can process new issues you create and search the web for relevant resources.

Lastly, you're going to need $5-10. $5 is the smallest amount you can pay to get an Anthropic API key, and we'll be using Anthropic's models to create the agent. The remaining $5 will easily cover the serverless AWS resources we'll be using. The agent will only cost a few cents each time it runs, so $5-10 dollars will go a long way.

With that out of the way, let's get started!

Creating the agent​

Generating a project​

To kick things off, we need to generate a new project. Since we'll be deploying our agent using AWS in Part 2, let's use the AWS CDK CLI for this. Don't worry if you're unfamiliar with the AWS CDK and its CLI. For now, just treat it like any other scaffolding tool.

Install the AWS CDK CLI with npm.

npm install -g aws-cdk

If you'd like, you can verify that it is installed correctly by checking its version.

cdk --version

At the time of this writing, the latest version is "2.1025.0 (build 8be6aad)".

Once installed, create a new directory and use the CDK CLI to generate the scaffold for a new TypeScript project.

mkdir linear-agent
cd linear-agent
cdk init app --language typescript

Besides the cdk.json file, this should look like a pretty standard Node.js / TypeScript project, with the bin directory containing the app's entrypoint and the lib directory containing the source code.

linear-agent
├── bin
│   └── linear-agent.ts
├── lib
│   └── linear-agent-stack.ts
├── node_modules
│   └── ...
├── test
│   └── linear-agent.test.ts
├── cdk.json
├── jest.config.js
├── package-lock.json
├── package.json
├── README.md
└── tsconfig.json

Creating an API key​

Before we can use Anthropic's models, we'll need an Anthropic API Console account and API key. To do this, simply create an account and then create a new API key.

Once you create an API key, copy its value and add it to a new .env file.

.env
ANTHROPIC_API_KEY=your_api_key

Then, update your project's .gitignore file so that you don't accidentally commit your API key 😅

.gitignore
*.js
!jest.config.js
*.d.ts
node_modules

# CDK asset staging directory
.cdk.staging
cdk.out

.env

Hello, Anthropic​

Now that we have a project setup and an API key, let's see just how easy it is to interact with Anthropic's latest models programmatically using their SDK.

Start by installing the Anthropic SDK as a dependency.

npm i @anthropic-ai/sdk

Then, create a lib/linear-agent.ts file with the following contents.

lib/linear-agent.ts
import Anthropic from "@anthropic-ai/sdk";

export class LinearAgent {
  private readonly client: Anthropic;

  constructor(apiKey: string) {
    this.client = new Anthropic({ apiKey });
  }
}

This file defines a class for our agent called LinearAgent that is instantiated with just your Anthropic API key. Using that API key, it creates an instance of the Anthropic client and stores it.

Next, let's define a simple method that will use this client to print a nice, warm greeting to the console.

lib/linear-agent.ts
import Anthropic from "@anthropic-ai/sdk";

export class LinearAgent {
  private readonly client: Anthropic;

  constructor(apiKey: string) {
    this.client = new Anthropic({ apiKey });
  }

  async greet(person: string) {
    // 1. Send message using Anthropic client and wait for a response
    const message = await this.client.messages.create({
      model: "claude-3-5-haiku-latest",
      max_tokens: 1024,
      messages: [
        {
          role: "user",
          content: `Give a nice, warm greeting to ${person}`,
        },
      ],
    });

    // 2. Create variable to collect text output
    let modelOutput = "";

    // 3. Collect text output
    for (const content of message.content) {
      switch (content.type) {
        case "text":
          modelOutput += content.text;
      }
    }

    // 4. Print output to console
    console.log(modelOutput);
  }
}
  • On line 13, we're telling the Anthropic client that we want to use the claude-3-5-haiku-latest model. This model is fast and cost effective, perfect for a simple task like this. In general, you want to start with the cheapest model you have access to. You can always choose a more powerful model if you're unhappy with how the cheaper one is performing.
  • On lines 27-32, we're looping over the content of the message returned by the Anthropic model and collecting it into a string we can print to the console. For our use case, we only care about the "text" content type, but there are a number of other content types that can be returned that can be useful for building more complex agents.

Now, create a simple run-agent.ts file with the following contents.

run-agent.ts
import { LinearAgent } from "./lib/linear-agent";

const apiKey = process.env.ANTHROPIC_API_KEY;

if (!apiKey) {
  console.error("Missing ANTHROPIC_API_KEY");
  console.debug("Did you add your API key to the .env file?");
  console.debug("Did you pass the --env-file=.env flag to tsx?");
  process.exit(1);
}

const agent = new LinearAgent(apiKey);
agent
  .greet("a friendly stranger") // that's you, hopefully :)
  .catch((err) => console.error(`Oops, something went wrong: ${err}`));

We won't need this file in the future when the agent is deployed to AWS, but we can use it to test locally for now. Using tsx, we can load the ANTHROPIC_API_KEY into the environment and execute the run-agent.ts file without needing to worry about building it first.

npx tsx --env-file=.env run-agent.ts

Hi there! How are you doing today? It's nice to meet you.

Voilà!

Unless you forgot to set your Anthropic API key, you should see a greeting like the one above printed to your console, courtesy of Anthropic's Claude Haiku 3.5 model.

However, run the script a few times and you may start to notice a problem...

Here's a pleasant greeting:

"Hi there! How are you doing today? It's nice to meet you."

Or, my personal favorite 😂

Smiles warmly and makes gentle eye contact

Hi there! How are you doing today? It's nice to meet you.

Large language models love to talk, and it can be quite challenging to get them to output exactly what you want. That might be okay when you're building a chatbot, but it can be very problematic when you're trying to work with an agent's output programmatically.

Luckily, there is an awesome framework called Zod that we can use to help rein in the model.

info

You can view usage and cost in the Anthropic Console. Something as simple as this greet(person:) method running on the Claude Haiku 3.5 model costs only fractions of a penny.

Defining schemas​

Zod makes it easy to create schemas in TypeScript that can be used to validate JSON data. By defining schemas that represent the model's input and its desired output, we can more easily enforce how the model responds. Zod will let us know if the model outputs anything that doesn't align with our schema. Then, we can simply tell the model what it did wrong and ask it to try again.

To start using Zod, we first need to install it as a dependency.

npm i zod

Once installed, we can define our schemas. For our agent, the input will be a new issue. In its simplest form, an issue can be represented by just a title and a description.

Example of an issue you might find in a project's backlog
{
  "title": "Add CSRF prevention",
  "description": "The purpose of this story is to prevent attackers from being able to trick a user into performing an unwanted action on a trusted site."
}

Let's define a schema that can represent this in a new file called lib/models/linear-issue.ts.

lib/models/linear-issue.ts
import z from "zod";

export const LinearIssue = z.object({
  title: z.string(),
  description: z.string(),
});

export type LinearIssue = z.infer<typeof LinearIssue>;
  • On line 3, we're exporting a ZodObject called LinearIssue. This is the actual schema. We'll eventually pass it to our model so that it can understand the structure of the its input.
  • On line 8, we're using the schema itself to define a type. We'll use this to add types to our method parameters. Yay TypeScript!

Next, we need to define a schema that represents the model's desired output. For our agent, the output should be an object containing an array of resources that the model believes are relevant to the given issue.

Example of the output we'd like our agent to produce
{
  "resources": [
    {
      "title": "Cross-Site Request Forgery Prevention Cheat Sheet",
      "url": "https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html",
      "description": "OWASP guide for preventing Cross-Site Request Forgery attacks."
    },
    {
      "title": "Cross-site request forgery (CSRF) - MDN Web Docs",
      "url": "https://developer.mozilla.org/en-US/docs/Web/Security/Attacks/CSRF",
      "description": "Mozilla developer documentation covering CSRF attack mechanics and defenses."
    }
  ]
}

Let's create another new file for this schema, this time called lib/models/resources.ts.

lib/models/resources.ts
import z from "zod";

export const Resources = z.object({
  resources: z.array(
    z.object({
      title: z.string(),
      url: z.string(),
      description: z.string(),
    }),
  ),
});

export type Resources = z.infer<typeof Resources>;

Just like the LinearIssue schema, we're using Zod to create a schema object and a TypeScript type, this time called Resources.

Using schemas​

With these schemas defined, we have everything we need to replace our greet(person:) method with one that's a bit more fitting.

lib/linear-agent.ts
import Anthropic from "@anthropic-ai/sdk";
import { LinearIssue } from "./models/linear-issue";

export class LinearAgent {
  private readonly client: Anthropic;

  constructor(apiKey: string) {
    this.client = new Anthropic({ apiKey });
  }

  async findRelevantResources(issue: LinearIssue) {
    // 1. Send message using Anthropic client and wait for response
    const message = await this.client.messages.create({
      model: "claude-3-5-haiku-latest",
      max_tokens: 1024,
      messages: [
        {
          role: "user",
          content: JSON.stringify(issue),
        },
      ],
    });

    // 2. Create variable to collect text output
    let modelOutput = "";

    // 3. Collect text output
    for (const content of message.content) {
      switch (content.type) {
        case "text":
          modelOutput += content.text;
      }
    }

    // 4. Print output to console
    console.log(modelOutput);
  }
}

Now, instead of asking the model to create a greeting, we're sending it an issue as a JSON string.

But what is the model supposed to do with that JSON string? Well, that's up to us. We need to tell it what we want using a system prompt. Let's add one now.

lib/linear-agent.ts
import Anthropic from "@anthropic-ai/sdk";
import z from "zod";
import { LinearIssue } from "./models/linear-issue";
import { Resources } from "./models/resources";

export class LinearAgent {
  private readonly client: Anthropic;

  constructor(apiKey: string) {
    this.client = new Anthropic({ apiKey });
  }

  async findRelevantResources(issue: LinearIssue) {
    // 1. Send message using Anthropic client and wait for response
    const message = await this.client.messages.create({
      model: "claude-3-5-haiku-latest",
      max_tokens: 1024,
      system: `
        You are a senior software engineer and tech lead on an agile
        delivery team.

        You will be given an issue from your team's backlog in the
        following format:
        ${JSON.stringify(z.toJSONSchema(LinearIssue))}

        Your responsibility is to search online for a handful of relevant
        technical documentation, tutorials, articles, libraries, or tools
        that would be beneficial to whomever is assigned to work on the
        issue.

        For each resource, you must provide a title, a url, and a one
        sentence description that summarizes the resource.

        You should output an unformatted JSON string representing an
        object that conforms to the following JSON schema:
        ${JSON.stringify(z.toJSONSchema(Resources))}

        It is important that you only return the JSON string without any
        newlines or formatting. Do not output any other text,
        explanations, or comments.
      `,
      messages: [
        {
          role: "user",
          content: JSON.stringify(issue),
        },
      ],
    });

    // 2. Create variable to collect text output
    let modelOutput = "";

    // 3. Collect text output
    for (const content of message.content) {
      switch (content.type) {
        case "text":
          modelOutput += content.text;
      }
    }

    // 4. Print output to console
    console.log(modelOutput);
  }
}

Using our schemas and Zod's toJSONSchema(schema:) method, this system prompt defines the structure of the input the model can expect, the structure of the output we want it to produce, and simple instructions that define what the model needs to do in order to produce that output.

If you're curious, this is what the result of z.toJSONSchema(Resources) looks like. It's a JSON representation of our Resources schema, and we simply stringify it to include in the system prompt.

JSON schema representation of Resources
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "resources": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "title": { "type": "string" },
          "url": { "type": "string" },
          "description": { "type": "string" }
        },
        "required": ["title", "url", "description"],
        "additionalProperties": false
      }
    }
  },
  "required": ["resources"],
  "additionalProperties": false
}

Next, let's use the Resources schema to validate the model's output. In addition, let's introduce a custom Result type like ones you might see in other strongly-typed languages like Rust or Swift. This type will help us avoid nesting a bunch of code in try / catch statements.

lib/linear-agent.ts
import Anthropic from "@anthropic-ai/sdk";
import z from "zod";
import { LinearIssue } from "./models/linear-issue";
import { Resources } from "./models/resources";

export type Result<T, E = Error> =
  | { ok: true; value: T }
  | { ok: false; error: E };

export class LinearAgent {
  private readonly client: Anthropic;

  constructor(apiKey: string) {
    this.client = new Anthropic({ apiKey });
  }

  async findRelevantResources(
    issue: LinearIssue,
  ): Promise<Result<Resources, string>> {
    // 1. Send message using Anthropic client and wait for response
    const message = await this.client.messages.create({
      model: "claude-3-5-haiku-latest",
      max_tokens: 1024,
      system: `
        You are a senior software engineer and tech lead on an agile
        delivery team.

        You will be given an issue from your team's backlog in the
        following format:
        ${JSON.stringify(z.toJSONSchema(LinearIssue))}

        Your responsibility is to search online for a handful of relevant
        technical documentation, tutorials, articles, libraries, or tools
        that would be beneficial to whomever is assigned to work on the
        issue.

        For each resource, you must provide a title, a url, and a one
        sentence description that summarizes the resource.

        You should output an unformatted JSON string representing an
        object that conforms to the following JSON schema:
        ${JSON.stringify(z.toJSONSchema(Resources))}

        It is important that you only return the JSON string without any
        newlines or formatting. Do not output any other text,
        explanations, or comments.
      `,
      messages: [
        {
          role: "user",
          content: JSON.stringify(issue),
        },
      ],
    });

    // 2. Create variable to collect text output
    let modelOutput = "";

    // 3. Collect text output
    for (const content of message.content) {
      switch (content.type) {
        case "text":
          modelOutput += content.text;
      }
    }

    // 4. Validate output and return it
    const result = this.parseModelOutput(modelOutput);

    if (result.ok) {
      return result;
    }

    return {
      ok: false,
      error: "Failed to find relevant resources",
    };
  }

  private parseModelOutput(
    modelOutput: string,
  ): Result<Resources, z.ZodError | unknown> {
    try {
      const modelOutputAsJSON = JSON.parse(modelOutput);
      const resources = Resources.parse(modelOutputAsJSON);

      return {
        ok: true,
        value: resources,
      };
    } catch (error) {
      if (error instanceof z.ZodError) {
        console.error("Model output does not conform to schema", error);
      } else {
        console.error("Model output is not valid JSON", error);
      }

      return {
        ok: false,
        error,
      };
    }
  }
}
  • On lines 6-8, we're defining the Rust-inspired Result type.
  • On lines 17-19, we're updating the findRelevantResources(issues:) method's return type to indicate that it returns a Result.
  • On lines 80-103, we're using this Result type to define a new helper method that makes it easy to determine if the model generated the expected output.
  • On lines 84-85, we're attempting to parse the model's output as a JSON object and then validating it against our Resources schema.
  • On lines 68-77, we're calling the new findRelevantResources(issues:) new helper method, checking if the result is OK, and then returning an appropriate result.

With a system prompt and validation now in place, let's update the run-agent.ts script so that we can see how things are progressing.

run-agent.ts
import { LinearAgent } from "./lib/linear-agent";

const apiKey = process.env.ANTHROPIC_API_KEY;

if (!apiKey) {
  console.error("Missing ANTHROPIC_API_KEY");
  console.debug("Did you add your API key to the .env file?");
  console.debug("Did you pass the --env-file=.env flag to tsx?");
  process.exit(1);
}

const agent = new LinearAgent(apiKey);
agent
  .findRelevantResources({
    title: "Add CSRF prevention",
    description:
      "The purpose of this story is to prevent attackers from being able to trick a user into performing an unwanted action on a trusted site.",
  })
  .then((result) => {
    if (result.ok) {
      console.log(result.value);
    } else {
      console.error(result.error);
    }
  })
  .catch((err) => console.error(`Oops, something went wrong: ${err}`));

Instead of calling the greet(name:) method, lines 14-25 now call the findRelevantResources(issue:) method. Once the promise resolves, we simply check the result to access the relevant resources or an error if something went wrong.

Now, run the agent again and let's see what we get.

npx tsx --env-file=.env run-agent.ts
{
  "resources": [
    {
      "title": "OWASP CSRF Prevention Cheat Sheet",
      "url": "https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html",
      "type": "documentation"
    },
    {
      "title": "Django CSRF Protection Documentation",
      "url": "https://docs.djangoproject.com/en/4.2/ref/csrf/",
      "type": "documentation"
    },
    {
      "title": "Spring Security CSRF Protection",
      "url": "https://docs.spring.io/spring-security/reference/features/exploits/csrf.html",
      "type": "documentation"
    },
    {
      "title": "Understanding CSRF Tokens",
      "url": "https://portswigger.net/web-security/csrf/tokens",
      "type": "tutorial"
    },
    {
      "title": "GitHub: Double Submit Cookie Pattern Implementation",
      "url": "https://github.com/pillarjs/understanding-csrf",
      "type": "code example"
    }
  ]
}

As Anakin would say, "It's working...IT'S WORKING!"

...

Or is it?

Let's explore the Dark Side of models for a moment. They love to make things up. We've asked our model to search online for relevant resources, and it's happily given us some. They are quite convincing, and by some miracle, the URLs it gave me when I ran this actually seem to work.

But here's the thing...we haven't given the model the ability to search the web yet. The output it just generated was entirely based on its training data. At best, it will give us links to resources that are slightly outdated. At worst, it will just make things up entirely.

For the model to be able to search the web, we need to give it the proper tool.

Using tools​

Expanding a model's capabilities is done by giving it access to additional tools. In our case, we need to give it a tool that it can use to search the web. Luckily for us, searching the web is such a common use case that we don't need to build this tool ourselves.

lib/linear-agent.ts
import Anthropic from "@anthropic-ai/sdk";
import z from "zod";
import { LinearIssue } from "./models/linear-issue";
import { Resources } from "./models/resources";

export type Result<T, E = Error> =
  | { ok: true; value: T }
  | { ok: false; error: E };

export class LinearAgent {
  private readonly client: Anthropic;

  constructor(apiKey: string) {
    this.client = new Anthropic({ apiKey });
  }

  async findRelevantResources(
    issue: LinearIssue,
  ): Promise<Result<Resources, string>> {
    // 1. Send message using Anthropic client and wait for response
    const message = await this.client.messages.create({
      model: "claude-3-5-haiku-latest",
      max_tokens: 1024,
      system: `
        You are a senior software engineer and tech lead on an agile
        delivery team.

        You will be given an issue from your team's backlog in the
        following format:
        ${JSON.stringify(z.toJSONSchema(LinearIssue))}

        Your responsibility is to search online for a handful of relevant
        technical documentation, tutorials, articles, libraries, or tools
        that would be beneficial to whomever is assigned to work on the
        issue.

        For each resource, you must provide a title, a url, and a one
        sentence description that summarizes the resource.

        You should output an unformatted JSON string representing an
        object that conforms to the following JSON schema:
        ${JSON.stringify(z.toJSONSchema(Resources))}

        It is important that you only return the JSON string without any
        newlines or formatting. Do not output any other text,
        explanations, or comments.
      `,
      messages: [
        {
          role: "user",
          content: JSON.stringify(issue),
        },
      ],
      tools: [
        {
          name: "web_search",
          type: "web_search_20250305",
          max_uses: 1,
        },
      ],
    });

    // 2. Create variable to collect text output
    let modelOutput = "";

    // 3. Collect text output
    for (const content of message.content) {
      switch (content.type) {
        case "text":
          modelOutput += content.text;
      }
    }

    // 4. Validate output and return it
    const result = this.parseModelOutput(modelOutput);

    if (result.ok) {
      return result;
    }

    return {
      ok: false,
      error: "Failed to find relevant resources",
    };
  }

  private parseModelOutput(
    modelOutput: string,
  ): Result<Resources, z.ZodError | unknown> {
    try {
      const modelOutputAsJSON = JSON.parse(modelOutput);
      const resources = Resources.parse(modelOutputAsJSON);

      return {
        ok: true,
        value: resources,
      };
    } catch (error) {
      if (error instanceof z.ZodError) {
        console.error("Model output does not conform to schema", error);
      } else {
        console.error("Model output is not valid JSON", error);
      }

      return {
        ok: false,
        error,
      };
    }
  }
}

And that's it. Really. Lines 54-60 are all we need to give the model the ability to use the web search tool.

There are other useful tools that are just as easy to use. Like the bash tool for running shell commands or the text editor tool for modifying text files. You can also write your own tools. For example, you could create a tool that exposes another AI agent to your model. This is a great way to get models with different personas to work together. But that's a topic for a future tutorial.

Run the agent again and you should see output similar to before; however, this time, the agent is actually searching for resources online instead of just making stuff up.

info

Using the web search tool incurs an additional cost but not a large one. You are only billed for each search, not the number of results returned. As of this writing, the web search tool costs $10 per 1,000 searches. Since we're limiting our model to a single search, it will cost $10 to search for relevant resources for 1,000 issues. Not too bad.

Retrying on error​

At this point, we have a fully functioning agent that can search the web for resources, and we're using Zod to enforce that the model's output is correct.

But what if the validation fails? What if the model outputs something that isn't valid JSON or doesn't conform to our Resources schema? Right now, we just use our Result type to return an error outcome. Let's update this so that we don't waste a web search we already paid for.

First, let's create a new variable called messages to maintain context between invocations of the model. This is required because models are stateless, meaning that you must provide them with all of the context they need each time they are called. If we want the model to correct itself, we'll need to provide it with the entire conversation history.

lib/linear-agent.ts
import Anthropic from "@anthropic-ai/sdk";
import z from "zod";
import { LinearIssue } from "./models/linear-issue";
import { Resources } from "./models/resources";

export type Result<T, E = Error> =
  | { ok: true; value: T }
  | { ok: false; error: E };

export class LinearAgent {
  private readonly client: Anthropic;

  constructor(apiKey: string) {
    this.client = new Anthropic({ apiKey });
  }

  async findRelevantResources(
    issue: LinearIssue,
  ): Promise<Result<Resources, string>> {
    const messages: Anthropic.Messages.MessageParam[] = [
      {
        role: "user",
        content: JSON.stringify(issue),
      },
    ];

    // 1. Send message using Anthropic client and wait for response
    const message = await this.client.messages.create({
      model: "claude-3-5-haiku-latest",
      max_tokens: 1024,
      system: `
        You are a senior software engineer and tech lead on an agile
        delivery team.

        You will be given an issue from your team's backlog in the
        following format:
        ${JSON.stringify(z.toJSONSchema(LinearIssue))}

        Your responsibility is to search online for a handful of relevant
        technical documentation, tutorials, articles, libraries, or tools
        that would be beneficial to whomever is assigned to work on the
        issue.

        For each resource, you must provide a title, a url, and a one
        sentence description that summarizes the resource.

        You should output an unformatted JSON string representing an
        object that conforms to the following JSON schema:
        ${JSON.stringify(z.toJSONSchema(Resources))}

        It is important that you only return the JSON string without any
        newlines or formatting. Do not output any other text,
        explanations, or comments.
      `,
      messages,
      tools: [
        {
          name: "web_search",
          type: "web_search_20250305",
          max_uses: 1,
        },
      ],
    });

    // 2. Create variable to collect text output
    let modelOutput = "";

    // 3. Collect text output
    for (const content of message.content) {
      switch (content.type) {
        case "text":
          modelOutput += content.text;
      }
    }

    // 4. Validate output and return it
    const result = this.parseModelOutput(modelOutput);

    if (result.ok) {
      return result;
    }

    return {
      ok: false,
      error: "Failed to find relevant resources",
    };
  }

  private parseModelOutput(
    modelOutput: string,
  ): Result<Resources, z.ZodError | unknown> {
    try {
      const modelOutputAsJSON = JSON.parse(modelOutput);
      const resources = Resources.parse(modelOutputAsJSON);

      return {
        ok: true,
        value: resources,
      };
    } catch (error) {
      if (error instanceof z.ZodError) {
        console.error("Model output does not conform to schema", error);
      } else {
        console.error("Model output is not valid JSON", error);
      }

      return {
        ok: false,
        error,
      };
    }
  }
}

Now, we need to create a loop so that we can append a new message to this messages array when the model doesn't output what we expect. However, the findRelevantResources(issue:) method is already a bit long, and a large portion of the it is just invoking the model and collecting its output.

Let's clean this up a bit with a new invokeModel(messages:) method. Doing so will make our code easier to read when we introduce a loop in the next step.

lib/linear-agent.ts
import Anthropic from "@anthropic-ai/sdk";
import z from "zod";
import { LinearIssue } from "./models/linear-issue";
import { Resources } from "./models/resources";

export type Result<T, E = Error> =
  | { ok: true; value: T }
  | { ok: false; error: E };

export class LinearAgent {
  private readonly client: Anthropic;

  constructor(apiKey: string) {
    this.client = new Anthropic({ apiKey });
  }

  async findRelevantResources(
    issue: LinearIssue,
  ): Promise<Result<Resources, string>> {
    const messages: Anthropic.Messages.MessageParam[] = [
      {
        role: "user",
        content: JSON.stringify(issue),
      },
    ];

    const modelOutput = await this.invokeModel(messages);
    const result = this.parseModelOutput(modelOutput);

    if (result.ok) {
      return result;
    }

    return {
      ok: false,
      error: "Failed to find relevant resources",
    };
  }

  private async invokeModel(messages: Anthropic.Messages.MessageParam[]) {
    // Send message using Anthropic client and wait for response
    const message = await this.client.messages.create({
      model: "claude-3-5-haiku-latest",
      max_tokens: 1024,
      system: `
        You are a senior software engineer and tech lead on an agile
        delivery team.

        You will be given an issue from your team's backlog in the
        following format:
        ${JSON.stringify(z.toJSONSchema(LinearIssue))}

        Your responsibility is to search online for a handful of relevant
        technical documentation, tutorials, articles, libraries, or tools
        that would be beneficial to whomever is assigned to work on the
        issue.

        For each resource, you must provide a title, a url, and a one
        sentence description that summarizes the resource.

        You should output an unformatted JSON string representing an
        object that conforms to the following JSON schema:
        ${JSON.stringify(z.toJSONSchema(Resources))}

        It is important that you only return the JSON string without any
        newlines or formatting. Do not output any other text,
        explanations, or comments.
      `,
      messages,
      tools: [
        {
          name: "web_search",
          type: "web_search_20250305",
          max_uses: 1,
        },
      ],
    });

    // Create variable to collect text output
    let modelOutput = "";

    // Collect text output
    for (const content of message.content) {
      switch (content.type) {
        case "text":
          modelOutput += content.text;
      }
    }

    return modelOutput;
  }

  private parseModelOutput(
    modelOutput: string,
  ): Result<Resources, z.ZodError | unknown> {
    try {
      const modelOutputAsJSON = JSON.parse(modelOutput);
      const resources = Resources.parse(modelOutputAsJSON);

      return {
        ok: true,
        value: resources,
      };
    } catch (error) {
      if (error instanceof z.ZodError) {
        console.error("Model output does not conform to schema", error);
      } else {
        console.error("Model output is not valid JSON", error);
      }

      return {
        ok: false,
        error,
      };
    }
  }
}

Finally, we need to introduce a loop. This loop will allow us to invoke the model muliple times until it produces correct output or until we reach some maximum number of attempts.

Remember, models are stateless, so each interaction needs to include the entire conversation history. Therefore, if the model fails on the first attempt, subsequent calls needs to include our original message containing the issue, the model's previous response that we couldn't parse, and new message telling the model what it did wrong.

lib/linear-agent.ts
import Anthropic from "@anthropic-ai/sdk";
import z from "zod";
import { LinearIssue } from "./models/linear-issue";
import { Resources } from "./models/resources";

export type Result<T, E = Error> =
  | { ok: true; value: T }
  | { ok: false; error: E };

export class LinearAgent {
  private readonly client: Anthropic;
  private readonly MAX_ATTEMPTS: number;

  constructor(apiKey: string, maxAttempts = 2) {
    this.client = new Anthropic({ apiKey });
    this.MAX_ATTEMPTS = maxAttempts;
  }

  async findRelevantResources(
    issue: LinearIssue,
  ): Promise<Result<Resources, string>> {
    const messages: Anthropic.Messages.MessageParam[] = [
      {
        role: "user",
        content: JSON.stringify(issue),
      },
    ];

    for (let attempt = 0; attempt < this.MAX_ATTEMPTS; attempt++) {
      const modelOutput = await this.invokeModel(messages);
      const result = this.parseModelOutput(modelOutput);

      if (result.ok) {
        return result;
      }

      messages.push({
        role: "assistant",
        content: modelOutput,
      });

      messages.push({
        role: "user",
        content: `
          Error while parsing your last message:
          ${result.error}

          You should output an unformatted JSON string representing an
          object that conforms to the Resources JSON schema you were
          given. Do not output any other text, explanations, or comments.
        `,
      });
    }

    return {
      ok: false,
      error: "Failed to find relevant resources",
    };
  }

  private async invokeModel(messages: Anthropic.Messages.MessageParam[]) {
    // Send message using Anthropic client and wait for response
    const message = await this.client.messages.create({
      model: "claude-3-5-haiku-latest",
      max_tokens: 1024,
      system: `
        You are a senior software engineer and tech lead on an agile
        delivery team.

        You will be given an issue from your team's backlog in the
        following format:
        ${JSON.stringify(z.toJSONSchema(LinearIssue))}

        Your responsibility is to search online for a handful of relevant
        technical documentation, tutorials, articles, libraries, or tools
        that would be beneficial to whomever is assigned to work on the
        issue.

        For each resource, you must provide a title, a url, and a one
        sentence description that summarizes the resource.

        You should output an unformatted JSON string representing an
        object that conforms to the following JSON schema:
        ${JSON.stringify(z.toJSONSchema(Resources))}

        It is important that you only return the JSON string without any
        newlines or formatting. Do not output any other text,
        explanations, or comments.
      `,
      messages,
      tools: [
        {
          name: "web_search",
          type: "web_search_20250305",
          max_uses: 1,
        },
      ],
    });

    // Create variable to collect text output
    let modelOutput = "";

    // Collect text output
    for (const content of message.content) {
      switch (content.type) {
        case "text":
          modelOutput += content.text;
      }
    }

    return modelOutput;
  }

  private parseModelOutput(
    modelOutput: string,
  ): Result<Resources, z.ZodError | unknown> {
    try {
      const modelOutputAsJSON = JSON.parse(modelOutput);
      const resources = Resources.parse(modelOutputAsJSON);

      return {
        ok: true,
        value: resources,
      };
    } catch (error) {
      if (error instanceof z.ZodError) {
        console.error("Model output does not conform to schema", error);
      } else {
        console.error("Model output is not valid JSON", error);
      }

      return {
        ok: false,
        error,
      };
    }
  }
}
  • On lines 12-16, we're creating a new variable called MAX_ATTEMPTS to limit the number of retries.
  • On line 29, we're introducing a new loop to so that we can try again if the result isn't what we expect.
  • On lines 37-52, we're adding new messages to the conversation. The first is a copy of the model's last output and the second details what was wrong with it. These will be sent to the model in the next iteration of the loop, assuming the maximum number of attempts haven't already been made.

Wrapping Up​

And with that, we've completed our AI agent! I hope this served as a good introduction to building AI agents and gets you excited about the possibilities of building agents that solve other interesting problems.

Be on the lookout for parts 2 and 3 of this tutorial. Part 2 will focus on deploying the agent we just made to AWS using a serverless architecture and infrastructure as code. Part 3 will focus on connecting the agent to Linear so that the agent will automatically start searching for revelant resources whenever new issues are created.