The latest census data shows the lack of investment in social and affordable housing has left Victoria in a state of peril.