メインコンテンツに移動

Barclays built a system to spot AI agents that might go rogue

Research setup ‘reads the mind’ of LLM-based agents and flags if they’re about to do something risky

「キルスイッチ」

Quants at Barclays have built a prototype monitoring system to read the minds of AI agents and catch them before they do anything too risky.

Ariye Shater and Hariom Tatsat borrowed from research into so-called mechanistic interpretability to address what Shater calls “tool risk” in agentic systems – the risk that AI agents call software tools in ways that are wasteful, risky or damaging, such as

コンテンツを印刷またはコピーできるのは、有料の購読契約を結んでいるユーザー、または法人購読契約の一員であるユーザーのみです。

これらのオプションやその他の購読特典を利用するには、info@risk.net にお問い合わせいただくか、こちらの購読オプションをご覧ください: http://subscriptions.risk.net/subscribe

現在、このコンテンツをコピーすることはできません。詳しくはinfo@risk.netまでお問い合わせください。

Sorry, our subscription options are not loading right now

Please try again later. Get in touch with our customer services team if this issue persists.

New to Risk.net? View our subscription options

Most read articles loading...

You need to sign in to use this feature. If you don’t have a Risk.net account, please register for a trial.

ログイン
You are currently on corporate access.

To use this feature you will need an individual account. If you have one already please sign in.

Sign in.

Alternatively you can request an individual account here