Further, “AIs can’t hold a copyright” just means the person who prompted the AI owns the work

Assuming you’re in the US: not true. In order to be able to be copyrighted, a work must have had its traditional elements of authorship produced by a human, and it has been well established that simply providing a prompt does not qualify.

Such a work has no copyright protections of its own, but that does not prevent it from being a violation of someone else’s copyright or trademark. If you’re responsible for the creation of a work, it’s irrelevant whether that work is itself copyrighted when determining if you’ve created a derivative work that infringes the copyright of the original rights-holder.

From https://copyright.gov/ai/ai_policy_guidance.pdf

in 2018 the Office received an application for a visual work that the applicant described as “autonomously created by a computer algorithm running on a machine.” 7

The application was denied because, based on the applicant’s representations in the application, the examiner found that the work contained no human authorship. After a series of administrative appeals, the Office’s Review Board issued a final determination affirming that the work could not be registered because it was made “without any creative contribution from a human actor.” 8

More recently, the Office reviewed a registration for a work containing human-authored elements combined with AI-generated images. In February 2023, the Office concluded that a graphic novel9 comprised of human-authored text combined with images generated by the AI service Midjourney constituted a copyrightable work, but that the individual images themselves could not be protected by copyright. 10

…

In the Office’s view, it is well-established that copyright can protect only material that is the product of human creativity. Most fundamentally, the term “author,” which is used in both the Constitution and the Copyright Act, excludes non-humans.

…

And in the current edition of the Compendium, the Office states that “to qualify as a work of ‘authorship’ a work must be created by a human being” and that it “will not register works produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author.” 22

…

In the case of works containing AI-generated material, the Office will consider whether the AI contributions are the result of “mechanical reproduction” or instead of an author’s “own original mental conception, to which [the author] gave visible form.” 24 The answer will depend on the circumstances, particularly how the AI tool operates and how it was used to create the final work.25 This is necessarily a case-by-case inquiry.

If a work’s traditional elements of authorship were produced by a machine, the work lacks human authorship and the Office will not register it 26 For example, when an AI technology receives solely a prompt 27 from a human and produces complex written, visual, or musical works in response, the “traditional elements of authorship” are determined and executed by the technology—not the human user. Based on the Office’s understanding of the generative AI technologies currently available, users do not exercise ultimate creative control over how such systems interpret prompts and generate material. Instead, these prompts function more like instructions to a commissioned artist—they identify what the prompter wishes to have depicted, but the machine determines how those instructions are implemented in its output. 28 For example, if a user instructs a text-generating technology to “write a poem about copyright law in the style of William Shakespeare,” she can expect the system to generate text that is recognizable as a poem, mentions copyright, and resembles Shakespeare’s style. 29 But the technology will decide the rhyming pattern, the words in each line, and the structure of the text. 30 When an AI technology determines the expressive elements of its output, the generated material is not the product of human authorship.31 As a result, that material is not protected by copyright and must be disclaimed in a registration application.32

In other cases, however, a work containing AI-generated material will also contain sufficient human authorship to support a copyright claim. For example, a human may select or arrange AI-generated material in a sufficiently creative way that “the resulting work as a whole constitutes an original work of authorship.” 33 Or an artist may modify material originally generated by AI technology to such a degree that the modifications meet the standard for copyright protection. 34 In these cases, copyright will only protect the human-authored aspects of the work, which are “independent of ” and do “not affect” the copyright status of the AI-generated material itself. 35

This policy does not mean that technological tools cannot be part of the creative process. Authors have long used such tools to create their works or to recast, transform, or adapt their expressive authorship. For example, a visual artist who uses Adobe Photoshop to edit an image remains the author of the modified image, 36 and a musical artist may use effects such as guitar pedals when creating a sound recording. In each case, what matters is the extent to which the human had creative control over the work’s expression and “actually formed” the traditional elements of authorship.37