MCP servers let an LLM use tools that live in other services, like your issue tracker or your
team's docs. AshAi.ToolLoop can call those tools right alongside the tools from your Ash actions.
In this guide, we'll take the chat that mix ash_ai.gen.chat generates and teach it to use MCP
servers, with each user connecting their own accounts. Along the way, we'll point out what the
generated code is doing for you, so that you can do the same thing in a tool loop of your own.
Goals
In this guide we will:
- Generate a chat, along with storage for each user's MCP credentials
- See what an MCP server's tools look like
- Give the chat the tools from the servers a user has connected
- Add a page where users connect and disconnect servers
- See how the chat handles a server that needs the user, and ask users before tool calls run
Requirements
You'll need a Phoenix application with Ash, AshPostgres and AshAuthentication, with a user
resource. If you don't have one yet, the
quick start for mix ash_ai.gen.chat will
create one for you.
This just gives you a starting point
You will almost certainly need to build quite a bit more, around things especially like managing MCP servers, tool approvals etc.
Generate the chat
mix ash_ai.gen.chat --user MyApp.Accounts.User --live
This gives us conversation and message resources, an Oban job that writes each reply with
AshAi.ToolLoop.stream/2, and a chat page at /chat. The rest of this guide is changes to the
code it generates, which is yours to change however you like.
Already have a generated chat?
Chats generated before ash_ai supported MCP servers can't wait on the user. The easiest way to
see what changed is to generate a chat in a scratch app and compare. Look at the awaiting,
resume_requested and approvals attributes and the :resume action on messages, the Respond
change, and the markup under replies.
Generate storage for credentials
MCP servers that act on a user's behalf generally use OAuth. Each user has to authorize your app,
and we have to keep the tokens we get back somewhere safe. mix ash_ai.gen.mcp_oauth generates
all of that for us:
mix ash_ai.gen.mcp_oauth --user MyApp.Accounts.User
mix ash.migrate
This adds:
- Resources for each user's tokens, encrypted with
ash_cloak MyApp.McpAuth.options/1, which returns theAshAi.Mcp.OAuthoptions for a given userMyAppWeb.McpAuthController, with routes that send a user off to authorize an MCP server (/mcp-auth/authorize), bring them back (/mcp-auth/callback), and disconnect them (/mcp-auth/connection)
It works as-is in development and test. In production you'll need to set a CLOAK_KEY, which we
cover at the end. If you'd rather keep credentials somewhere else, see the
MCP Client Authorization guide.
What do MCP tools look like?
Before we wire anything up, let's look at what we actually get from an MCP server. Open
iex -S mix and connect to DeepWiki's MCP server, which is public, so it doesn't need
authorization:
{:ok, conn} = AshAi.Mcp.Client.connect(name: "deepwiki", url: "https://mcp.deepwiki.com/mcp")
{:ok, definitions} = AshAi.Mcp.Client.list_tools(conn)
Enum.map(definitions, & &1["name"])
# => ["ask_wiki_question", "read_wiki_contents", "read_wiki_structure"]Those are the server's tool definitions, exactly as it sent them. To hand them to a tool loop, we turn them into ReqLLM tools:
tools = AshAi.Mcp.Client.req_llm_tools(conn, definitions)
Enum.map(tools, fn {tool, _callback} -> tool.name end)
# => ["mcp__deepwiki__ask_wiki_question", "mcp__deepwiki__read_wiki_contents",
# "mcp__deepwiki__read_wiki_structure"]Each one is a {tool, callback} pair. The names are prefixed with the server's name, so tools
from different servers can't collide, and when the model calls one, its callback calls the tool on
the server. You can call a tool yourself, too:
{:ok, result} =
AshAi.Mcp.Client.call_tool(conn, "read_wiki_structure", %{"repoName" => "ash-project/ash"})
result.content
# => [%{"type" => "text", "text" => "Available pages for ash-project/ash: ..."}]Those {tool, callback} pairs are exactly what AshAi.ToolLoop takes as :extra_tools, and for
a public server, that's really all there is to it. The rest of this guide is about servers that
need to know who the user is.
Give the chat MCP tools
For a server that uses OAuth, we give it AshAi.Mcp.OAuth as its auth, along with the options
for the user. Every request to that server then carries the user's token, which is refreshed for
us when it expires.
Let's add a module that lists the MCP servers our chat can use, and builds the tools from the ones a user has connected:
# lib/my_app/chat/mcp_servers.ex
defmodule MyApp.Chat.McpServers do
@moduledoc "The MCP servers the chat can use."
require Logger
# The model sees each server's tools as `mcp__<name>__<tool>`
@servers [
[name: "linear", url: "https://mcp.linear.app/mcp"]
]
def all, do: @servers
def server(config, user) do
AshAi.Mcp.Client.server(
name: config[:name],
url: config[:url],
auth: {AshAi.Mcp.OAuth, MyApp.McpAuth.options(user.id)},
allowed_private_hosts: MyApp.McpAuth.allowed_private_hosts()
)
end
@doc "Tools from the servers `user` has connected, for `AshAi.ToolLoop`'s `extra_tools`."
def tools(user) do
Enum.flat_map(@servers, fn config ->
with {:ok, conn} <- AshAi.Mcp.Client.connect(server(config, user)),
{:ok, definitions} <- AshAi.Mcp.Client.list_tools(conn) do
AshAi.Mcp.Client.req_llm_tools(conn, definitions)
else
# Not connected yet, or the server no longer accepts their token
{:error, {kind, _info}} when kind in [:authorization_required, :insufficient_scope] ->
[]
{:error, _reason} ->
Logger.warning("Couldn't list the tools of the #{config[:name]} MCP server")
[]
end
end)
end
endListing a server's tools needs the user's token too, so a server the user hasn't connected doesn't contribute any tools. The model won't know it exists until they connect it.
Now we pass these to the tool loop. Each reply is written by the generated Respond change, in an
Oban job that runs as the user who sent the message. That means context.actor is the user whose
tokens we want. Build the tools at the start of the reply, and pass them as :extra_tools:
# lib/my_app/chat/message/changes/respond.ex
Ash.Changeset.before_transaction(changeset, fn changeset ->
message = changeset.data
mcp_tools = MyApp.Chat.McpServers.tools(context.actor)
...
|> AshAi.ToolLoop.stream(
...
approvals: approvals(message.approvals),
extra_tools: mcp_tools
)There's one more spot to update in the same function. The change stores a fingerprint of the system prompt and the tools with each reply. When an earlier reply is replayed, its reasoning is only kept if the fingerprint still matches, since reasoning is tied to the tools it was produced with. So the MCP tools need to be part of the fingerprint too:
fingerprint =
Transcript.fingerprint(
system_prompt(),
AshAi.list_tools(
otp_app: :my_app,
tools: true,
actor: context.actor,
tenant: context.tenant
) ++ Enum.map(mcp_tools, fn {tool, _callback} -> tool end)
)Let users connect
Users need somewhere to connect their accounts. Let's add a page that lists our servers, with a Connect or Disconnect button for each. The buttons are just forms that post to the routes we generated earlier.
# lib/my_app_web/live/mcp_connections_live.ex
defmodule MyAppWeb.McpConnectionsLive do
use MyAppWeb, :live_view
require Ash.Query
on_mount {MyAppWeb.LiveUserAuth, :live_user_required}
def mount(_params, _session, socket) do
user = socket.assigns.current_user
# Tokens are stored under each server's canonical URL
connected =
MyApp.McpAuth.Token
|> Ash.Query.filter(user_id == ^user.id)
|> Ash.Query.select([:server])
|> Ash.read!(authorize?: false)
|> MapSet.new(& &1.server)
servers =
for config <- MyApp.Chat.McpServers.all() do
{:ok, resource} = AshAi.Mcp.OAuth.canonical_resource(config[:url])
%{name: config[:name], url: config[:url], connected?: resource in connected}
end
{:ok, assign(socket, :servers, servers)}
end
def render(assigns) do
~H"""
<div class="mx-auto max-w-lg p-8">
<h1 class="mb-4 text-lg font-semibold">Connections</h1>
<div :for={server <- @servers} class="flex items-center justify-between border-b py-3">
<span>{server.name}</span>
<.form :if={!server.connected?} for={%{}} action={~p"/mcp-auth/authorize"} method="post">
<input type="hidden" name="server_url" value={server.url} />
<input type="hidden" name="return_to" value={~p"/connections"} />
<button class="btn btn-sm btn-primary">Connect</button>
</.form>
<.form :if={server.connected?} for={%{}} action={~p"/mcp-auth/connection"} method="delete">
<input type="hidden" name="server_url" value={server.url} />
<input type="hidden" name="return_to" value={~p"/connections"} />
<button class="btn btn-sm">Disconnect</button>
</.form>
</div>
</div>
"""
end
endThe tokens are stored under each server's canonical URL, which is why we compare against
AshAi.Mcp.OAuth.canonical_resource/1 instead of the URL as we wrote it.
Then add the route next to the chat's, inside the ash_authentication_live_session:
# lib/my_app_web/router.ex
live "/connections", McpConnectionsLiveTry it out
Start your server and visit /connections. Press Connect, and you'll be sent to the MCP server to
sign in and approve your app, then brought back. Your app registers itself with the server's
authorization server the first time, so there's nothing to set up for each server.
Now head over to /chat and ask for something that needs one of the server's tools. You'll see
each tool call as a badge on the reply.
When a server needs the user
Sometimes a server stops accepting a user's access in the middle of a reply. They might have revoked it, or a tool might need more permissions than they granted. The generated chat already handles this, so there's nothing to change here, but it's worth knowing what's going on.
When this happens, the MCP tool doesn't fail. Instead, it asks the tool loop to pause. The reply stops, and shows a Connect button for that server along with a Continue button. The user connects, comes back to the chat, and presses Continue. The tool call that needed them runs, and the reply picks up where it left off. If they send another message instead, the buttons go away, and the model is told that the call didn't run.
If a server stops accepting a user's access between replies, there's nothing to pause. Its tools
are left out of the next reply, and /connections shows Connect for it again. AshAi refreshes
tokens before they expire, so this only happens when a server actually stops accepting them.
In your own tool loop
A paused loop returns an AshAi.ToolLoop.Result with status: :paused, and each call that needs
the user in paused_calls, as %{call: call, reason: {kind, info}}. kind is
:authorization_required or :insufficient_scope, and info has the server_url and the
server's challenge in www_authenticate.
Send the user to the URL from AshAi.Mcp.OAuth.authorize_url/3, passing
challenge: info.www_authenticate so that they're asked for what the server wants. Once they're
back, run the loop again with result.messages. The call was left unanswered, so it runs first.
The generated chat stores what a reply is waiting on in its awaiting attribute. Continue runs
the message's :resume action, which runs the Respond change again with the reply added to the
history.
Ask before tool calls
MCP servers can do things on the user's behalf, like creating issues or sending messages, so you'll probably want users to approve some calls before they run. The generated chat already has Approve and Deny buttons for this. It just doesn't ask about anything yet, because which calls need approval is up to you.
In the Respond change, uncomment approve_tool_call: and return :ask for the calls to
approve:
# lib/my_app/chat/message/changes/respond.ex
approve_tool_call: fn
%{name: "mcp__" <> _}, _context -> :ask
_call, _context -> :ok
end,Now the reply stops before those calls, and shows them with Approve and Deny buttons. Approving runs them, and denying tells the model the user said no. If an approved call then needs the user to reconnect, they won't be asked to approve it again.
The function gets each call's name and arguments, so you can be as specific as you like. For
example, to let reads through and ask about everything else:
approve_tool_call: fn
%{name: "mcp__linear__" <> tool}, _context when tool in ["list_issues", "get_issue"] -> :ok
%{name: "mcp__" <> _}, _context -> :ask
_call, _context -> :ok
end,In your own tool loop
Returning :ask stops the loop with status: :awaiting_approval, and the calls in
pending_approvals. To continue, run it again with an :approvals map from call ids to
:approve or {:deny, reason}. Calls that were approved and then paused come back in
Result.approvals. Store them and pass them back, so the user isn't asked twice.
Let the model read resources
Besides tools, MCP servers can offer resources: files, records, or other content, each with a URI.
Some tools also return links to resources instead of their contents. To let the model list and
read a server's resources, pass resources: true when building its tools:
AshAi.Mcp.Client.req_llm_tools(conn, definitions, resources: true)That adds two tools, mcp__linear__list_resources and mcp__linear__read_resource. Text comes
back as text, and images as images, for models that accept them in tool results. MCP leaves it to
the app to decide which resources the model sees, so only turn this on for servers whose resources
you're happy for it to read.
The approval rule above asks about these too, since their names start with mcp__. They only
read, so you may want to let them through:
approve_tool_call: fn
%{name: "mcp__linear__" <> tool}, _context
when tool in ["list_resources", "read_resource", "list_issues", "get_issue"] ->
:ok
%{name: "mcp__" <> _}, _context ->
:ask
_call, _context ->
:ok
end,You can read resources yourself, too, for example to attach one a user picked to their message:
{:ok, [resource | _]} = AshAi.Mcp.Client.list_resources(conn)
{:ok, result} = AshAi.Mcp.Client.read_resource(conn, resource["uri"])
# Text, or content parts when the resource includes images
{:ok, content} = AshAi.Mcp.Client.format_resource(result)Disconnecting
Disconnecting from /connections revokes the user's tokens at the authorization server, if it
supports revocation, and deletes them.
If you delete users, call MyApp.McpAuth.disconnect_all(user.id) first. Deleting a user deletes
their tokens along with them, but only revoking them ends the grants on the authorization servers'
side.
Going to production
Set
CLOAK_KEY. This is what encrypts the stored tokens. You can generate one with32 |> :crypto.strong_rand_bytes() |> Base.encode64(). Don't lose it! If you do, every user has to connect again.Serve your app over HTTPS at its public URL. The callback URL is built from your endpoint's
urlconfig, which Phoenix sets fromPHX_HOST. Over HTTPS, your app also identifies itself to authorization servers with a Client ID Metadata Document, served at/mcp-auth/client-metadata.json.Register by hand where you have to. A few authorization servers don't let apps register themselves. Register your app with them yourself, with
https://your-app.example.com/mcp-auth/callbackas its redirect URI, and add the credentials toMyApp.McpAuth.options/1:pre_registered: %{ "https://mcp.example.com/mcp" => [ issuer: "https://auth.example.com", client_id: System.fetch_env!("EXAMPLE_CLIENT_ID"), client_secret: System.fetch_env!("EXAMPLE_CLIENT_SECRET") ] }Allow private hosts on purpose. Your app only connects to public addresses, so that a URL from a server can't be used to reach services on your network. If you want to use an MCP server on your own network, add its host to
config :my_app, :mcp_allowed_private_hosts. Development allowslocalhost.
You can run all of this on as many nodes as you like. AshAi doesn't keep anything in memory between requests, and token refreshes are coordinated through the database.
What next?
- Keep tools stable across a conversation. The tools are listed at the start of every reply,
so if a server changes its tools, they change mid-conversation, and you lose prompt caching. You
can store the definitions from
AshAi.Mcp.Client.list_tools/1when a conversation starts, and build the tools from those instead. See "Tool definitions and prompt caching" inAshAi.Mcp.Client. - Use servers that take an API key. Pass
headers:toAshAi.Mcp.Client.server/1instead ofauth:. - Keep credentials somewhere else. See the MCP Client Authorization guide for writing your own store.