More than 130 affordable homes could be built on a disused former banking site if plans are agreed with the council.