Software complexity in industrial automation systems continues to increase, which makes the programming of Programmable Logic Controllers (PLCs) a repetitive and time-consuming engineering task. Traditional programming of IEC 61131-3 code requires domain expertise and manual effort. This study explores the potential of Large Language Models (LLMs) to automate parts of this process by generating PLC code directly from natural-language specifications. This article introduces a systematic evaluation framework that allows a reproducible comparison and rating of different LLMs and prompting strategies. With the proposed methodology, state-of-the-art LLMs such as Gemini 2.5 Pro, DeepSeek V3, and Grok V3 are evaluated regarding their capability to generate PLC programs in Structured Text (ST) and Function Block Diagram (FBD) format. The generation process uses the PLCopen XML format as target format, enabling direct import and verification in CODESYS V3. The article defines quantitative quality metrics, like import success rate, compilation rate, and functional pass rate, to objectively assess the syntactic and semantic correctness of the generated programs.
The results demonstrate that LLMs are already capable of generating functional PLC programs for a wide range of control tasks, particularly when using ST. FBD generation, by contrast, remains more error-prone due to the inherent graphical structure and the need to maintain numerous XML-based references and connections. The application of optimized prompting strategies such as Few-Shot and Chain-of-Thought (CoT) shows significant improvements in both compilation and pass rates, highlighting that structured problem decomposition enhances LLM reasoning for engineering tasks. Beyond benchmarking, the developed framework provides reusable prompt templates, quantitative evaluation metrics, and a set of validated test cases for future research and industrial application. The findings underline that the integration of LLMs into PLC development workflows can substantially accelerate engineering cycles, reduce human error, and enable knowledge-based automation. At the same time, the article identifies current limitations, particularly in handling graphical languages and complex modular architectures.