The temple is mentioned in various ancient texts, including the Markandeya Purana, Ashoka inscriptions, and the travel accounts of early explorers. According to a nearby inscription, the temple was established in the 8th century AD around a clay idol. It is commonly believed that King Suratha, who had lost much of his kingdom in battle and was betrayed by his ministers, founded the temple. He took refuge in the forests near the current location of the temple under the guidance of the sage Sumedha. King Suratha is said to have personally carved the clay idol of Shri Rajarajeshwari and performed penance to regain his kingdom, offering his own jewel-studded crown to the deity. The idol itself is believed to be up to 5000 years old.
Over time, numerous inscriptions referencing the temple were recorded but have since been lost due to neglect. However, some surviving inscriptions have been found in Kariyangala village, Ammunje, and within the temple itself, and are now preserved by the Karnataka Government.
The region surrounding the temple was governed by several dynasties, including the Kadamba, Chalukya, Alupa, Rashtrakoota, Hoisala, Vijayanagara, Ikkeri, and Mysore. Many of these dynasties invested considerable resources into the temple and donated agricultural lands for its upkeep. The Alupa dynasty, which ruled between 710 AD and 720 AD, is particularly noted for its contributions to the temple and the promotion of Shri Rajarajeshwari worship in the area. Later, Queen Chennammaji of Keladi visited the temple and presented it with a grand chariot.