Is PVP the only stabilizer for the particles? Often times the stabilizer will actually inhibit assembly, as their purpose to stop aggregation. You create particles of a desired size by adding the stabilizer after a given time period, or in the case of a polymer stabilizer such as PVP, the polymer governs the actual particle size. If you want gold particles to aggregate (assemble) you might want to try some type of chemical linker, such as a dithiol.
edit: spelling