Terence Tao put it plainly: there is no evidence that LLMs exhibit genuine creativity.
Yes, they have solved some Erdős problems. But these are low-hanging fruit, questions that attracted little attention and that yield once the right existing techniques are applied. That is not