Kataragama is one of the few places in Sri Lanka which is worshipped by most of the popular religious people in Sri Lanka. According to the Buddhists, Katragama is one of the 16 places visited by Buddha during his 3 visits to Sri Lanka.

The history of Kataragama goes back to pre Christian era and both Buddhist and Hindu literature have identified this place in various ways. According to chronicles some of the people who accompanied Vijaya from India in 543 BC established a village called Kajara–Gama which is thought to be current Kataragama.

According to popular legends, Elara the Indian invader who captured then capital of Anuradhapura sent a spy called Kadira to the south (Rohana) to spy on the Sinhalese Royals. He established himself in Kataragama and brought his Indian wife Thevani with him. Later he fell in love with a beautiful 16 year old local girl called Valli and married her with the consent of the previous wife.

Kadira is said to have operated 6 spy rings again sub divided in to 12 groups. It was these groups which later was transformed in to six heads and 12 arms of the Kataragama Deviyo. When the Elara was finally defeated by King Dutugamunu (161–131 BC), Kadira the a spy master became a respected citizen in the area of Kataragama. He specially helped the Sinhalese people who accepted him as a benefactor. He was so respected that when he died the Sinhalese people elevated him to a deity and built a shrine (devale) for him. But the Tamils in Kataragama never accepted him as a deity and ignored him. Kadira was known as the "Kadira Deviyo" who later became Kataragama Deviyo. Those Tamils who worshipped at the shrine introduced their own god Skanda Kumar as the reigning deity of this shrine.

Some legends consider the Katragama Deviyo (deity) as the powerful regional warrior king Mahasena who ruled the area when the Buddha made his 3rd visit to Sri Lanka in the 8th year of his enlightenment (580 BC). This warrior king took to Buddhism after meeting the Buddha and was elevated to a deity by the community after his death. It is believed that Kiri Vehera has been built by him on the place that the Buddha meditated. Some believe that the Kiri Vehera was built by the Ksastriyas who accompanied King Vijaya (the first descendent of the Sinhalese race) or by prince Aggabodhi, King Devanpiyathissa’s (250–210 BC) younger brother’s son who ruled Magama during 3rd century BC. It is said that the rulers in this area were present at the ceremonial planting of the Sri Maha Bodhi sapling at Anuradhapura at the invitation of the King Devanmpiyathissa (250–210 BC). Later one of the first eight saplings of this tree was planted in Kataragama which is worshipped up to date by all Buddhists.

According to Hindu beliefs god Skanda is also known as Murugan, Arumugam, Kandasami (Skanda Swami), Subrahmanya, etc. According to Hindu legends God Skanda came to Sri Lanka after a row with his wife Thevani and landed in the southern part of the island. He made Vedihiti Kanda near Kataragama his adobe. One day he saw a beautiful 16 year old local girl called Valli who was adapted by the veddha chieftain of the tribe who lived in the area. Skanda unable to win her love consulted his brother god Ganesh to help him. Finding out that Valli feared elephants, a plan was devised for Ganesh to appear as a Elephant and Skanda to come to her rescue. Before turning in to the elephant, Ganesh gave a pot of water to Skanda and asked him to pore the water on to him after the planed attack so he would turn back to the Human form. When Skanda approached Valli in form of a old man she was chocking on some food and in the eagerness to help her, he dropped all the water. At the same time Ganesh appears in form of a elephant and scares her. Skanda offers her to help with the condition of marring him. Valli having no choice consents to marry him and then he appears in his true form. But unfortunate Ganesh, has to stay with a elephant head since Skanda has dropped the water.

For Islamists, Kataragama is associated with Al–Khidr meanning "The Green (one)", though Muslim commentators are not agreed on who he exactly was. Some say he is a prophet while others say he is a wali meaning one who is close to God or saint.

The Veddas (aka Wannniye Aetto) call him the Kande Yaka (The spirit of the mountain) and claims to have been visiting Kataragama for the last thousands of years. They are believed be the indigenous people of the country who walked to the island thousands of years ago when from the Indian sub continent when Sri Lanka was connected to India over the Adams bridge.