There aren't many words in English containing the letter combination 'gyp'. There's gypsum (the mineral from which plaster of Paris is made) and gypsophila (a flower). The origin of both those words is the Greek word for 'chalk'. Gypsy (also spelt gipsy), however, has another history. It comes from the word Egyptian, as gypsies were once believed to have come from Egypt.
The 'E' was dropped from the word Egyptian to finally arrive at 'gypsy'. This linguistic process of dropping an initial, unstressed vowel sound is known as aphesis. Other examples include esquire which became squire and acute which dropped the 'a' to become cute.