Jesse wants to give Becky singing lessons because he thinks she can't sing. During their wedding they both sang the "Forever" song. She has a beautiful voice. She also sang "What I Did for Love" with Danny during a telethon and was on-key and did fine.
When Becky (off key) joins Jesse, Danny and Joey in "House On Pooh Corner", Jesse is seen giving her a silent (yet closed mouth) look of horror while the background audio has him still singing the song.