Good morning, AI enthusiast.

Anthropic is significantly enhancing Claude's utility, unveiling new Integrations that embed the AI directly into users' apps and data streams via the MCP protocol. This update also includes an advanced Research capability leveraging these connections.

The push aims to transform Claude into a central workflow partner, potentially eliminating tedious context-switching. As AI assistants become more deeply embedded in our tools, will this interconnectedness truly unlock the next level of productivity and streamline complex tasks?

In today’s AI recap:

• Claude’s new app Integrations and Research feature

• Microsoft releases Phi-4 reasoning SLMs

• LM Arena benchmark faces controversy

Claude Gets Connected

The Recap: Anthropic just supercharged Claude, launching Integrations that connect the AI assistant to your apps and data using the remote Model Context Protocol (MCP). They also rolled out an advanced Research capability that leverages these new connections.

Unpacked:

• Integrations allow Claude to securely access data and take actions within popular tools like Jira, Confluence, Asana, Intercom, Linear, Sentry, Square, PayPal, Cloudflare, and thousands more via partners like Zapier, turning Claude into a more informed collaborator across your workflow.

• The enhanced Research feature now conducts deeper investigations, potentially taking up to 45 minutes to synthesize information from the web, Google Workspace, and your connected Integrations into comprehensive, cited reports.

• Developers can expand Claude's reach even further by building and hosting their own remote MCP servers, with resources like Cloudflare offering templates and hosting solutions to simplify the process.

• Integrations and advanced Research are now in beta for Claude Max, Team, and Enterprise plans (Pro coming soon), while web search is globally available on all paid plans. Check Anthropic's help center for getting started details.

Bottom line: This is a significant step towards making AI assistants truly integrated workflow partners. By connecting Claude directly to where work happens, Anthropic aims to save you hours of manual context switching and boost productivity, paving the way for a more interconnected AI ecosystem.

Microsoft's Phi-4 Gets Reasoning

Microsoft announced new additions to its Phi family of small language models (SLMs), unveiling open-weight reasoning models designed to deliver impressive performance on complex tasks without the scale of frontier models.

Unpacked:

• The new lineup includes Phi-4-reasoning and Phi-4-reasoning-plus, both 14-billion parameter models trained using supervised fine-tuning and reinforcement learning techniques on Phi-4.

• Despite their smaller size, the models rival much larger models like DeepSeek-R1-Distill-Llama-70B and approach the performance of the massive 671B DeepSeek-R1 on demanding math and science benchmarks, according to detailed technical reports.

• Microsoft attributes the strong performance to a data-centric approach, leveraging curated data and synthetic reasoning traces derived from models like OpenAI's o3-mini, along with structured outputs to enhance coherence.

• Released under a permissive MIT license, the models are available on Hugging Face and compatible with popular inference frameworks like vLLM and llama.cpp, promoting broad accessibility.

Bottom line: This release reinforces the trend that smaller, smartly trained models can achieve powerful reasoning capabilities. It makes advanced AI reasoning more accessible for developers and applications operating under resource or latency constraints, potentially lowering the barrier to entry for sophisticated AI features.

LM Arena Benchmark Backlash

The Recap: The popular Chatbot Arena leaderboard faces scrutiny after a study alleged preferential private testing for top AI labs like Meta and OpenAI, potentially skewing results. LM Arena, the org behind the benchmark, refutes the claims, calling the study inaccurate.

Unpacked:

• The study alleges LM Arena let labs like Meta privately test numerous models (up to 27 variants for Meta before Llama 4) and only publish top scores, effectively gaming the rankings.

• Researchers also claim favored labs received more "battle" exposure on the platform, potentially collecting extra data that could improve performance on related benchmarks like Arena Hard by up to 112%.

• LM Arena counters that pre-release testing is a known practice outlined in their public policy, arguing it helps the community access frontier models and doesn't constitute unfair treatment if one provider tests more than another.

• While the study relied partly on self-identification for private models, the authors call for more transparency, like clear limits on private tests and public disclosure of all scores, suggestions LM Arena has pushed back on.

Bottom line: This controversy highlights the growing pains of AI benchmarking and the critical need for transparency, especially as evaluators like LM Arena gain influence. With LM Arena recently becoming a company, maintaining trust and perceived impartiality will be crucial for its success and the community relying on its rankings.

The Shortlist

NVIDIA achieved a new benchmark in automatic speech recognition with its Parakeet-v2 model, showing a 25% improvement in word error rate across multiple languages.

Pinterest introduced automatic "AI modified" labels for AI-generated images and a filter to help users see fewer AI pins in certain categories.

Google expanded access to its AI Mode in Search, removing the waitlist for Labs users and adding product/place cards plus a history panel.

Signing off,

David, Lucas, Mitchell — The Recap editorial team