At the very start of Kermit's song, you can see the background is really a set since the top part on the right is cut off, revealing part of the wall. As the song continues, you can also see the bottom part of the set.
Throughout the song, you can see Kermit's legs hanging over are fake and not attached to the puppet's body; plus, Kermit's armsock can be seen as well.