Americans are becoming much less optimistic about the economy as growth slows