Long Jaafar established and developed his administrative center at Bukit Gantang and made Kuala Sungai Limau at Trong the principal harbour of the Larut Settlement. In 1857, Long Jaafar was succeeded by his son, Ngah Ibrahim. Sultan Jaafar Muazzam Shah presented an acknowledgement letter to Ngah Ibrahim on 24 May 1858. This letter was signed by Sultan Jaafar, Raja Muda Ngah Ali and the Raja Bendahara of Perak. In the time of Ngah Ibrahim, the Chinese increased in number and in early 1860, two large groups were formed by the Chinese, the “Five Associations” whose members worked in the mines of Klian Pauh and the “Four Associations” whose members worked in the mines of Klian Baharu.
Mining rights were given to the Hakka “Five Associations” or Go-Kuan (五館 or 五群) and the Cantonese “Four Associations” or Si-Kuan (四館). Chung Keng Quee (鄭景貴) was leader of the Hakka Go-Kuan and the Hai San (海山) society they belonged to and began to operate his tin mines in Larut in1860. Larut was destined to be plagued by four major wars between members of both the Ghee Hin Society (義興私會黨) that owned the Cantonese Go-Kuan and the Hakka Hai San society. Many Hakka fled China when the Taiping Rebellion broke out there and found work in the mines of Chung Keng Quee establishing his position over the mining area in Larut as leader of the Hai San from 1860 to 1884.
The capital of Perak was moved from Bandar Baru (New Town) to Taiping after Datok Maharaja Lela assassinated the first British Resident of Perak, Mr.James Wheeler Woodford Birch at Pasir Salak in 1875. In 1937, the capital of Perak was moved from Taiping to Ipoh.
The town’s mining industry continued to thrive; the country’s first railway was built to transport tin from Taiping to Port Weld (now known as Kuala Sepetang) at the coast for export. The first train in Malaysia to take its schedule was on June 1 1885.
By 1900, an English language school, a newspaper, and the Perak Museum (the oldest in Malaysia) had been established.