When Olivia Snow investigated Lensa and its avatar creation for Wired, she found that no matter how modest the photos she submitted to the app were, she always received avatars with "spectacularly sized breasts and full nudity"—including photos she used from her childhood. Those photos heavily influenced her photoset, she said, creating explicit content with child-like features.
While Lensa says not to submit photos of children to its "Magic Avatar" creator, there is nothing stopping people from doing so, just like nothing is stopping them from submitting pictures of other people, whether it be celebrities they fantasize about or ex's they want to get revenge on (via Wired).
According to Tech Crunch research into the app, Lensa makes creating AI images of literally anyone as simple as hitting "upload." In their tests of the app's artistic skills, Tech Crunch found that even with poorly edited photos of celebrity heads on naked bodies, Lensa delivered "higher quality" photos. This has led to companies like Unstable Diffusion being able to capitalize on AI-generated pornography.
On their website, Prisma Labs explains that the creators of the AI model they use, LAION-5B, have added an "NSWF filter," but sexualization does still unfortunately happen. Since LAION-5B is an AI model "trained on unfiltered internet content...it reflects the biases humans incorporate into the images they produce." Prisma adds that they're aware that their filters don't stop everything, which is why they don't advise minors to use their app.
