Software complexity in industrial automation systems continues to increase, which makes the programming of Programmable
Logic Controllers (PLCs) a repetitive and time-consuming engineering task. Traditional programming of IEC
61131-3 code requires domain expertise and manual effort. This study explores the potential of Large Language Models
(LLMs) to automate parts of this process by generating PLC code directly from natural-language specifications. This
article introduces a systematic evaluation framework that allows a reproducible comparison and rating of different LLMs
and prompting strategies. With the proposed methodology, state-of-the-art LLMs such as Gemini 2.5 Pro, DeepSeek V3,
and Grok V3 are evaluated regarding their capability to generate PLC programs in Structured Text (ST) and Function
Block Diagram (FBD) format. The generation process uses the PLCopen XML format as target format, enabling direct
import and verification in CODESYS V3. The article defines quantitative quality metrics, like import success rate, compilation
rate, and functional pass rate, to objectively assess the syntactic and semantic correctness of the generated programs.
The results demonstrate that LLMs are already capable of generating functional PLC programs for a wide range
of control tasks, particularly when using ST. FBD generation, by contrast, remains more error-prone due to the inherent
graphical structure and the need to maintain numerous XML-based references and connections. The application of optimized
prompting strategies such as Few-Shot and Chain-of-Thought (CoT) shows significant improvements in both compilation
and pass rates, highlighting that structured problem decomposition enhances LLM reasoning for engineering
tasks. Beyond benchmarking, the developed framework provides reusable prompt templates, quantitative evaluation metrics,
and a set of validated test cases for future research and industrial application. The findings underline that the integration
of LLMs into PLC development workflows can substantially accelerate engineering cycles, reduce human error,
and enable knowledge-based automation. At the same time, the article identifies current limitations, particularly in handling
graphical languages and complex modular architectures.