Ipoh Gardens Ltd shaped much of Sydney’s business district in the 1980’s and 1990’s.

The Queen Victoria Building was restored between 1984 and 1986 by the Malaysian Company Ipoh Ltd, a  subsidiary of IGB Berhad, at a cost of $86 million. 

During the restoration a car park station was built under York Street. The building's restoration retained its exemplary features including the trachyte stairs, tessellated tiled surfaces and column capitals and created a commercial establishment that houses high end fashion stores, cafés, and restaurants which reflect the original purpose of the building in the city of Sydney.

"If there is a lesson for heritage projects from this, it is that heritage buildings should not only be restored but should be put to a use that will make them freely accessible to the community at all times " Yap Lim Sen (Chairman, Ipoh Ltd Australia).

The site on which World Square now stands was originally home to the Anthony Hordern & Sons department store, a six-storey building which opened in 1905 on the former Brickfield Hill. The flagship store operated from 1905 until 1966 with the store converted to Horderns HomeWorld which operated until its closure in 1973. After the takeover of Anthony Horderns by Waltons in January 1970, the Brickfield Hill site was then sold to Stocks and Holdings Ltd, for $8.5 million, ending 64 years of retail pioneering and ownership of the building by Anthony Hordern's Ltd.

In the early 1980s the building's new owner, a Malaysian-based Ipoh Garden Development, planned to try to save the Anthony Hordern & Sons building. However it was found to be too costly after receiving independent advice from the CSIRO on the state of the building's concrete and other fittings. The building was controversially demolished in 1986 which led to the site remaining undeveloped throughout most of the 1990s due to planning disputes. During the dispute, scenes for the 1995 film Mighty Morphin Power Rangers: The Movie were shot on the former property of Hordern & Sons and future site of World Square.