Prince Charles was reportedly baffled when someone commented that he looked buff in an old photo (see the photo here).
Buff has a number of meanings in English. In fact, there are thirteen separate entries for the word in the OED - as noun, adjective, verb, and interjection. The meaning "Of a person (esp. a man) or their body: muscular, well-toned; physically attractive" is number 7 in the list. The first citation is from 1982 (the photo of Prince Charles was taken in 1986). The origin of this sense of the word is uncertain, according to the OED, but it suggests it is probably related to the verb to buff meaning 'to polish'. Buff is used about young women these days, too.
The earliest sense of the word is as a noun meaning a blow, stroke or buffet, and dates back to the 15th century. This sense is now obsolete, except in the game blind-man's-buff. Buff is a yellowish-brown colour and this meaning, plus the verb sense meaning 'to polish', are related. They both come from buff meaning buffalo.
There's more teen talk that parents often don't understand here.