Optimizing Light Source: Before and After Corrections for Realistic Pictures

2023-07-03 02:34:00
The comparison between after correction and before correction of the light source Preface This article discusses a problem that is often encountered when producing realistic or real-life pictures, that is, the light source is too bright. On many popular models, due to the use of too many pictures of daytime models or studio highlight pictures for training, unnatural hyper-highlights will appear when the characters are drawn, even though the place is set at night, or the entire photo is blurred. The problem that the shadow of the light source does not follow the layout of the hint word. The Stable Diffusion prompts introduced earlier — light and shadow and Stable Diffusion advanced — light and shadow control are both feasible light and shadow control methods, and this article is aimed at more subtle control, changing the light source and shadow of the generated picture again, And it produces different degrees of light and shade so that the producer can choose the best light and shadow results. Method We first use txt2img to generate a character picture in the night forest: (masterpiece, top quality, best quality, official art, beautiful and aesthetic: 1.3), extreme detailed, Hana, fighting stance, angry, cleaned teeth, floating hair, wearing white kimono and white cape, long sleeves, hand holding a wooden club (weapon), night, half moon, moonlight, fantasy, forest lora:Hana:0.4 Many times the pictures generated by the model do not necessarily follow our Think about the layout of light and shade and light and shadow, sometimes because too many models’ studios are used to take photos during model training, or LoRA lacks diverse light and shadow during training, or too many prompt words are used to restrain each other, resulting in some strong The light and shadow effect is canceled. In short, when you can’t produce the desired light and shadow after trying various prompt words, you can try to use ControlNet with Loopback to generate a whole batch of pictures with different levels of light and shade, and then pick the best result from them. First, in this case, I wanted to create a moon above the center of the frame as a light source. I open any drawing software, such as GIMP, open a new canvas, select a black background, and then use a brush to make a white light source mask, and input this picture into img2img, and the prompt words remain the same as the original picture: Then in img2img In the setting, use a medium weight like Denoising strength 0.4: Then turn on ControlNet and set up two groups. The first group uses Openpose to fix the poses of the characters when the picture is generated so that the use of ContorlNet is more important: ControlNet’s Openpose settings The second group of ControlNet can use canny (edge ​​detection) or normal (normal map), use To fix the shape and expression of the characters in the original picture, using ContorlNet is more important: Finally, the most important thing is the Loopback setting. With the Loopback loop input capability, we can slowly restore the entire black basic mask picture. Here I set 10 rounds, and the final denosing strength is set to a high weight of 0.75, which will produce a large change in light and dark in the next few rounds: After all the settings are completed, you can run the map, and wait for ten rounds to finish After that, you can choose the picture: It can be seen that at the beginning, AI can barely calculate the outline of the people near the light source under the dark mask, but as the result picture is continuously input, the picture will become clearer and clearer, and The shadow distribution will change according to the relative relationship of the objects and the strength of the light source. If you use ControlNet’s canny to fix the shape, some strong shadow boundaries will be sampled, thus interfering with the light and shadow distribution. If so, you can download Canny’s picture to erase these extra lines, or you can use normal to fix it shape, but may cause a complete change in clothing style. In the end, I chose a picture that best fit my imagination with moonlight and enough darkness in the night forest, about the seventh picture: The results of changing the brightness and light and shadow In addition to the shadow of the characters, the shadows of the forest and road in the background are also changed It is completely different, more in line with the atmosphere of moonlight lighting, and basically meets my needs! I wish you all a happy AI calculation!
1688352064
#Stable #Diffusion #advanced #progressive #light #shadow