>> Support engineer ran customer query through Claude (trained on our public and internal docs) and it very, very confidently made a bunch of stuff up in the response.
> Yeah, LLMs are not really good about things that can't be done.
From the GP's description, this situation was not a case of "things that can't be done", but instead was the result of a statistically generated document having what should be the expected result:
It was quite plausible sounding and it would have been
great if it worked that way, but it didn't.
The core issue is likely not with the LLM itself. Given sufficient context, instructions, and purposeful agents, a DAG of these will not produce such consistently wrong results with good grounding context
There are a lot of devils in the details and there are few in the story
> Yeah, LLMs are not really good about things that can't be done.
From the GP's description, this situation was not a case of "things that can't be done", but instead was the result of a statistically generated document having what should be the expected result: