In Cocomelon, children can learn the lowercase alphabet through fun and engaging songs. Each letter is associated with a specific word, such as "j" for "jel". This educational approach helps young learners become familiar with the alphabet in an entertaining way, promoting early literacy skills and language development.