Veracruz has a reputation as a town with a rich history but little to show for it. True enough, because much of that rich history involved sackings by pirates, heavy bombardment by three different foreign powers (including during the wonderfully named French Pastry War), and epidemics of malaria, yellow fever, and cholera.
One might not necessarily want to preserve such a history, even when those same events didn't destroy artifacts. For this reason, and for the character of the natives, Veracruz (unlike Puebla) is better suited to cafe goers than museum goers. With the exception of the old fort of San Juan de Ulúa and perhaps the aquarium, the museums can be missed. You come here for the feel of the tropics, the balmy air, and the carefree attitude of the locals.