Residing on the coast of East Africa, the Great Mosque was one of the many structures in Kilwa that was built in a fairly common fashion of its time. Like its name suggests it was remarkably larger than the other mosques being built on the island. The walls of the mosque were built and developed of generally squared and coursed coral stone. Often, the courses in between would have chips built in to level them out. These types of walls were barely made below the ground and were made with mud mortar. Columns also made of coral stone helped to support the high vaulted ceiling. The floors were slowly made of lime plaster, and later, the walls were made of lime mortar. There was also a separate domed chamber used for private worship by the sultan. However, due to decline in wealth in Kilwa in the late 14th century, it led to less construction of buildings and building maintenance because the demand for lime and building stone lessened. Because of Kilwa’s decline, the Great Mosque was reported to be in ruins. The mosque collapsed during the rule of Abu’l Mawahib al-Hasan bin Sulaiman.

The mosque is placed at the top of a small hill bridging to the qibla, making the mosque larger in width than it is length. The oblique arches in the interior support the structure of the ceiling. Later, another set of these arcades were built to reinforce them. The external edicule of a mihrab built in the walls is topped by an arch. The outside veneer of the qibla divider is strengthened by a 30 cm wide brace that finishes the establishment. This mihrab has a broken arch that resides on top of two columns with rectangular capitals.

The northern part of the Great Mosque is the oldest portion of the mosque, having been built between 1131 and 1170. All that remains of the northern part are walls whose establishments are made out of a genuinely standard mechanical assembly, quadrangular rubble in coral limestone. Today, the nine monochrome polygonal segments that hold up the leveled roof are still visible.

The 11th or 12th century saw the construction of the smaller northern prayer hall, which was part of the initial phase. A total of nine pillars, initially cut from coral but later covered in wood, supported a total of 16 bays. The building, which was totally covered in roofs, was possibly one of the earliest mosques to be constructed from the ground up without a courtyard. In the 13th century, side pilasters, timber, and transverse beams were added.

As time progressed, Kilwa had a growing trading economy and wealth flourished. Because of this growth in the economy, it led to a new extension built to the Great Mosque. During this time of prosperity, the Great Mosque was rebuilt and renovated many times leading to a majority of the structure being new at the time.