How Far Are LLMs from Symbolic Planners? An NLP-Based Perspective

arXiv:2508.01300 · cs.AI · Submitted 2025-08-02 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "How Far Are LLMs from Symbolic Planners? An NLP-Based Perspective".

Jane: Here's the gist of the paper. The gist The reasoning and planning abilities of Large Language Models (LLMs) are limited because their outputs often contain mistaken or hallucinated actions, making them unreliable as planners.

Tom: How it works The paper approaches planning with LLMs as a natural language processing (NLP) task, treating the plan as a form of language output given that LLMs are NLP models themselves.

Jane: The authors propose a recovery pipeline consisting of an NLP-based evaluation of the generated plans, along with three stages to recover the plans through NLP manipulation of the LLM-generated plans, and eventually complete the plan using a symbolic planner This pipeline provides a holistic analysis of LLM capabilities in the context of AI task planning, enabling a broader understanding of the quality of invalid plans.

Tom: Plan Evaluation and Validation The proposed evaluation pipeline includes NLP-based metrics to evaluate the quality of plans, which better capture the nature of LLM-generated plans.

Paper summary: Tom: So, to recap, the paper "How Far Are LLMs from Symbolic Planners? An NLP-Based Perspective" frames planning with LLMs as an NLP task and proposes a recovery pipeline to assess plan quality beyond just whether it hits the goal.

Jane: They focus on comparing the generated plans, called pi zero to a ground-truth plan, pi GT, and they use NLP-based metrics to assign quality labels like "one correct" or "five redundant".

Lu: They also talk about using Longest Common Subset and Longest Common Subsequence algorithms to quantify how similar the generated plan is structurally to the valid one.

Meng: That sounds like a lot of text processing overhead for something that should be simple, but they are trying to see the linguistic patterns in their errors.

Lalam: The paper claims this pipeline improves the success rate from twenty-one point nine percent to twenty-seven point five percent, which is an improvement of twenty-five percent for the successfully generated plans, and they also found that action quality doesn't get worse during generation, which is interesting because you'd think it should <ref:2508.01300#pg1>.

Tom: Wait, so they are getting a better success rate, but are those plans actually good enough to be trusted when we deploy them?

Jane: The authors argue that their pipeline provides novel insights into the potential of plans and helps us measure exactly where and how LLMs go wrong in generating plans, which is important for error analysis.

Lu: And they explicitly argue that LLMs fail at showing NLP-based evidence of reasoning required in planning algorithms, making them unreliable as planners.

Tom: That's a big statement then. So even with the recovery steps, the core issue seems to be a lack of underlying logical derivation during plan generation itself.

Meng: From my side, if it can't show that reasoning during generation, it means we can't really rely on the LLM to do the hard thinking part of planning alone.

Jane: The numbers they give are telling us that on average, for all non-empty plans, only three point five five actions per plan are executable, which is way less than the symbolically generated optimal plans that have an average length of nine point four seven actions.

Lalam: And their study also shows that NL oneshot performs best by most metrics even when its PDDL counterpart has a higher initial success rate, and the recovery pipeline is more effective for shorter ground-truth plans, meaning it really helps when the target plan is already relatively short.

Conclusion: Tom: So to wrap up this discussion on "How Far Are LLMs from Symbolic Planners? An NLP-Based Perspective", the main message is that success rate alone isn't a reliable measure for plan quality.

Jane: The paper shows that planning cannot be reliably done based solely on LLM reasoning; you still need a classical planner to improve the plan after the initial generation.

Lu: The authors conclude that their pipeline, even with all the recovery and refinement steps, still falls short of the quality and reliability of classical planners.

Meng: So what does this mean for deployment? It suggests that while LLMs can generate text that looks like a plan, they lack the core reasoning to ensure those actions are logically sound in a complex environment.

Lalam: The paper also points out that the quality of LLM plan generation might actually depend on the specific architecture of the models they are using.

Tom: It sounds like future work needs to focus more on evaluating how these models handle temporal constraints, which is something they mention as an area for future investigation.

Jane: Exactly. We need to adapt these evaluation components to properly test the LLMs' temporal ability in planning, because right now, relying only on their linguistic output isn't enough for safe autonomous action.

Ma’ayan Armony, Albert Merono-Peñnuela, Gerard Canal

King’s College London

cs.AI

Submitted: 2025-08-02

Updated: 2026-10-20

Code: https://github.com/maayan25/llm-plan-evaluation

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 73/100

The gist: The gist The reasoning and planning abilities of Large Language Models (LLMs) are limited because their outputs often contain mistaken or hallucinated actions, making them unreliable as planners.

Key concepts

LLMs in Planning
Large Language Models are used to generate plans from unstructured input like natural language. The paper treats this as an NLP task because LLMs are fundamentally NLP models. However, they struggle with logical constraints and action effects, making them unreliable for complex planning.
Recovery Pipeline
This is a multi-step process designed to fix flawed LLM plans. It involves three stages: first, using NLP metrics to assess plan quality; second, manipulating the generated text through intermediate plans ($\pi_i$); and finally, using a symbolic planner to generate a valid final plan ($\pi_4$).
Plan Quality Metrics
Instead of just checking if a plan succeeds (success rate), the authors use NLP metrics like LCS (Longest Common Subsequence) to compare an LLM's plan ($\pi_0$) against a correct ground-truth plan ($\pi_{GT}$). This allows for finer evaluation, labeling actions as 'correct,' 'misplaced,' or 'redundant.'
Symbolic Planning
This refers to the traditional method of planning that uses formal logic and structured descriptions (like PDDL) to ensure plans are logically sound. The paper uses a symbolic planner as the final step in its recovery pipeline because it provides the necessary logical consistency that LLMs lack.

Terminology

Summary

The gist The reasoning and planning abilities of Large Language Models (LLMs) are limited because their outputs often contain mistaken or hallucinated actions, making them unreliable as planners.

How it works

The paper approaches planning with LLMs as a natural language processing (NLP) task, treating the plan as a form of language output given that LLMs are NLP models themselves. The authors propose a recovery pipeline consisting of an NLP-based evaluation of the generated plans, along with three stages to recover the plans through NLP manipulation of the LLM-generated plans, and eventually complete the plan using a symbolic planner This pipeline provides a holistic analysis of LLM capabilities in the context of AI task planning, enabling a broader understanding of the quality of invalid plans.

Plan Evaluation and Validation

The proposed evaluation pipeline includes NLP-based metrics to evaluate the quality of plans, which better capture the nature of LLM-generated plans. This involves comparing each plan π0 to the corresponding ground-truth plan πGT, where SR is defined as the percentage of plans that are valid out of all instances in an experiment. The evaluation process assigns an action quality label based on similarity to actions in πGT, such as 1 correct, 2 misplaced, and 5 redundant. Plan similarity is further quantified using LCS algorithms, specifically the longest common subset and the longest common subsequence.

NLP Manipulation and Recovery

The recovery mechanism involves four intermediate plans, denoted as πi, where i is the level of refinement. These include π1, the NLP-transformed version which is most similar to πGT, and sub-plans like π2 and π3. A final improvement is achieved using a symbolic planner, yielding the plan π4, which represents the complete plan after recovery. The Steps to Validity (StV) metric quantifies the effort needed to transform πi into πGT by calculating edit distance between plans and assigning an equal cost to repair, add, and remove operations.

Findings on LLM Reliability

The findings reveal no clear evidence of underlying reasoning during plan generation, and that the pipeline still falls short of the quality and reliability of classical planners. On average, only the first 2.65 actions of the plan are executable, with the average length of symbolically generated plans being 8.4 actions. While the pipeline improves action quality and increases the overall success rate from 21.9% to 27.5%, this improvement is not sufficient to match classical planners. The analysis suggests that LLMs fail at showing NLP-based evidence of reasoning required in planning algorithms, making them unreliable as planners.

Model and Prompt Type Comparison

The evaluation compares plans generated by six families of LLMs across four prompt types and two PDDL domains. Table 1 shows that NL oneshot performs best by most metrics, even when its PDDL counterpart has a higher initial SR. Furthermore, the results show that the recovery pipeline is more effective for shorter GT plans (πGT denotes the plan’s length). The analysis indicates that action quality does not deteriorate during generation, which negates the possibility that LLMs start computing a high-quality plan and struggle to complete it successfully.

Conclusion

The paper concludes that success rate is insufficient as an indicator of plan quality. The results reinforce the notion that planning cannot be reliably done based solely on LLM reasoning, and a classical planner is thus required to improve the plan. The quality of LLM plan generation may be influenced by the models’ architectures. Future work may also involve adapting the evaluation components to support temporal constraints and evaluate LLMs’ temporal ability in planning.

--- Page 1 ---

How Far Are LLMs from Symbolic Planners? An NLP-Based Perspective

Abstract The reasoning and planning abilities of Large Language Models (LLMs) have been a frequent topic of discussion in recent years. Their ability to take unstructured planning problems as input has made LLMs’ integration into AI planning an area of interest. Nevertheless, LLMs are still not reliable as planners, with the generated plans often containing mistaken or hallucinated actions. Existing benchmarking and evaluation methods investigate planning with LLMs, focusing primarily on success rate as a quality indicator in various planning tasks. In this paper, we approach planning with LLMs as a natural language processing (NLP) task, given that LLMs are NLP models themselves. We propose a recovery pipeline consisting of an NLP-based evaluation of the generated plans, along with three stages to recover the plans through NLP manipulation of the LLM-generated plans, and eventually complete the plan using a symbolic planner. This pipeline provides a holistic analysis of LLM capabilities in the context of AI task planning, enabling a broader understanding of the quality of invalid plans. Our findings reveal no clear evidence of underlying reasoning during plan generation, and that a pipeline comprising an NLP-based analysis of the plans, followed by a recovery mechanism, still falls short of the quality and reliability of classical planners. On average, only the first 2.65 actions of the plan are executable, with the average length of symbolically generated plans being 8.4 actions. The pipeline still improves action quality and increases the overall success rate from 21.9% to 27.5%.

--- Page 2 ---

Introduction Large Language Models (LLMs) (Vaswani et al. 2017; Devlin et al. 2019; Brown et al. 2020) are deep neural architectures trained on large corpora to generate token sequences based on input prompts. Their outputs are derived through statistical pattern recognition rather than semantic understanding or reasoning over logical constraints. AI planning involves searching for a plan that achieves a specified goal in a set environment. The information required to solve the planning task, namely, the applicable actions, their effects and preconditions, the related objects, and the initial and goal states, is often expressed using the Planning Domain Definition Language (PDDL) (McDermott et al. 1998), as a structured description of the domain and the problem. Despite recent interest in applying LLMs to AI planning, and especially to robot task planning (Singh et al. 2023; Silver et al. 2024; Chen et al. 2024), LLMs still struggle with demonstrating reliable, accurate outcomes (Valmeekam et al. 2023a,b; Kokel et al. 2025a,b) due to a lack of reasoning abilities. As a consequence of their architecture and design, which aim to predict the next token reliably, LLMs do not evaluate the constraints and effects of actions. This evaluation is the backbone of symbolic planners and their ability to soundly and consistently produce successful plans. Lal et al. (2024) found that LMs are not notably better at predicting the order of actions than random choice (50%), and their output is not consistent with logical assumptions. They have also observed that changing the order of actions, when those are independent of each other, incorrectly changes the model’s response.

Improvements for AI systems

  1. Apply an NLP-based evaluation of generated plans to capture the nature of LLM-generated plans, moving beyond a binary success rate to assess how far the failing plans are from success. This provides a more nuanced analysis of linguistic and structural patterns in the output.

  2. Implement a plan recovery mechanism consisting of three stages: NLP-based evaluation, followed by NLP manipulation of the LLM-generated plans, and finally, complete the plan using a symbolic planner. This pipeline aims to improve action quality and increases the overall success rate from 21.9% to 27.5%.

  3. Use an Action Quality Mapping (AQM) to assign labels like correct, misplaced, and redundant to each action in an LLM-generated plan, allowing for position-based precedence and scoring based on semantic similarity and parameter alignment.

  4. Employ a hybrid plan recovery method that utilizes planning tools and NLP evaluation to improve the initial plan by identifying the best subset or subsequence, denoted as pi2 or pi3. This leads to plans where pi3 is found to be valid.

  5. Use a Step-by-Validity (StV) metric based on Levenshtein distance between plans and symbolic planning tools to quantify the effort needed for transformation, showing that pi1 requires only 2, indicating that this pi1 is indeed closer to being valid and optimal.

  6. Integrate state simulation using VAL to determine the Last Executable Action (LEA), which identifies actions that violate domain constraints, helping diagnose where LLMs fail to reason over constraints.

  7. Utilize plan potential scoring based on NLP transformations like circular shifts and consistent parameter remapping to select the best recovered plan, as seen in pi1 receiving a higher score of 25.6 compared to pi0's raw quality score of 18.53 with a penalty of 0.66.

  8. Use the results from the evaluation pipeline to inform model selection by noting that the improvement in SR is more significant for Qwen-1.5B than for Qwen-7B, suggesting a more granular approach to evaluating quality leads to better model choice.

Abstract

AI planning provides agents the ability to act autonomously towards achieving a goal in a given environment. Recent advances in Large Language Models (LLMs) have introduced the prospect of using unstructured input for planning, making LLMs in planning an active area of research, especially for supporting autonomous agents. However, LLMs remain unreliable as long-horizon planners, since some of the plans they produce contain mistaken or hallucinated actions; executing such plans could lead to unexpected and harmful outcomes. Existing benchmarks for LLMs in planning primarily rely on binary evaluation metrics, providing limited insight into the quality of plans that do not achieve the goal, and how far those are from becoming valid. In this paper, we frame the evaluation of LLMs in planning as a natural language processing (NLP) task, as LLMs are themselves NLP models. We propose a pipeline comprising three NLP-based steps that assess plan quality from complementary perspectives, followed by a symbolic planning step to achieve a valid plan, to provide a finer-grained understanding of the quality of invalid plans. In our experiments, we find no clear evidence of logical derivation or state-tracking during LLM plan generation -- on average, for all non-empty plans, only 3.55 actions per plan are executable, despite symbolically-generated optimal plans having an average length of 9.47 actions, which suggests that LLMs are still far from the reliability of symbolic planners. Our suite of metrics is further used to investigate patterns in LLM plan generation over time, and can provide a useful metric for assessing the quality of individual invalid plans.

Sources

Related papers