Azure AI Evaluation: Unable to track tool call and tool call accuracy for Azure AI Search

[m-gheini]

Describe the bug
I created an agent in Foundry with Azure AI Search as tool and tried evaluating that with similarity, tool call accuracy, and tool call success metrics. For tool call accuracy I am getting error: 100% of the batch run failed. 'str' object has no attribute 'get'. But I can see tool call in traces and for some queries I see 'Pass' for tool call success. Even though in one of the reasons for tool call success I can see tool call to azure ai search is getting mentioned, but evaluator is missing it and cannot validate it.

To Reproduce
Steps to reproduce the behavior:

1. Create an agent with Azure AI Search as tool.
2. Create evaluation for this agent with tool call accuracy as metric.

Expected behavior
Tool call gets recognized and validate if content was retrieved from index.

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @luigiw @needuv @singankit.