Automated software synthesis, text-to-code (code-generation), can be an important
aid to software developers. Developments in this area have existed practically since
the advent of computing. The new artificial intelligence methods, deep learning (DL),
and one type of them, large language models (LLMs) emerged in the last decade, represent
a new opportunity for creating this type of application. We survey the transformer
technology that underlies the operation of language models. Present paper briefly
reviews the topic of software synthesis and code generation, briefly mentioning the
main trends. The document concludes with an empirical study demonstrating the described
results, where the T5 language model is fine-tuned to generate SQL queries. A fine-tuning
set of a few thousand examples train T5 to produce quality queries. The economic principle
of diminishing returns (Gossen I)
applies, and it is not worth using too large a fine-tuning set. In fact, it is the
attractiveness of fine-tuning.
