According to the legend from the Shiv Purana, an ancient Hindu text Hindu text in Sanskrit, Lord Shiva was greatly disturbed after the death of his consort, Sati (Parvati). Overwhelmed with grief, Shiva decided not to part with her body, carrying with him wherever he went. Greatly distressed by such uncalled action, the other divine members of heaven cut her body into pieces and threw the pieces onto `Mrityulok`, the land of mortals. Wherever the parts of her body fell, it became Shaktipeeth, a centre of the female manifest form of cosmic energy, thereby rendering the place sacred. It is believed that Goddess Parvati`s elbow fell here in Ujjain temple site.

There is also an interesting tale in Skanda Purana about how Parvati acquired her epithet, Harsiddhi. Once when Shiva and Parvati were alone on Mount Kailash, two demons, Chand and Prachand tried to force their way in. Shiva called upon Chandi (Parvati) to destroy them, which she did. Pleased, Shiva conferred upon her the epithet of `one who vanquishes all` (harsiddhi).
In the centre of this ancient Hindu temple is a rock smeared with turmeric paste and vermilion, believed to represent the head of the Gupta King, Chandragupta Vikramaditya (reigned-380-415 AD), offered to Goddess Durga, on the eve of Dusshera. Shakti, the female principal of primal cosmic energy, had to be appeased with sacrifices. The temple has two unique pine-shaped iron lamp stands that loom to a height of 15 feet and display their radiance after being lit. Hundreds of lamps burning bright simultaneously make a magical sight, especially on Navratri, the nine-day festival celebrated in the month of October, dedicated to Goddess Durga. The red temple, an ancient Hindu structure beyond the lamp stands symbolizing the potency of Durga, is peculiar to the Maratha art of architecture.

Yet another arresting feature of the Harsiddhi Temple is the Sri Yantra, or nine triangles that represent nine names of Durga. A Yantra is symbolic of the cosmos and is used for meditation. Each division of a yantra is figurative of Shakti. Also enshrined in the temple are the illustrious dark vermilion image of Annapurna, the Goddess of Nourishment, seated next to Mahasaraswati, the Goddess of Wisdom and Knowledge.