Is Tree-of-Thought Prompting Strategy Better than Chain-of-Thought? Vaping Cessation Analysis Using Large Language Models.

Résumé

Vapingis gainingpopularity amongadolescents andposes severe risks to users. Social media platforms such as Reddit and X offer insights into user behaviors and attitudes regarding vaping. In previous studies, our team explored the ability of large language models (LLMs) to perform binary classification at a sentence level to determine if LLMs can be used to identify users for a vaping cessation application. Maintaining the same goal, this study expands to compare OpenAI's GPT-o1 and GPT-o3-mini, Google's Gemini 2.0 Flash and Gemma 2, Meta's LLAMA 3.3, Deepseek's R1, and xAI's Grok-2 against human annotators to examine which models best perform binary classification to identify quit intention and multiclass classification to detect quit stages. We tested these models with emerging and prompts together with a simple prompt to see which strategy performed best. To our knowledge, this is investigation of prompting. Our initial results indicate that tree-of-thought and chain-of-thought prompting do not boost performance.